-- Logs begin at Thu 2022-06-23 15:00:25 UTC, end at Thu 2022-06-23 16:17:43 UTC. -- Jun 23 15:46:17.989544 dv-cinder-835384-9-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jun 23 15:46:17.998237 dv-cinder-835384-9-iscsi devstack@keystone.service[80143]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jun 23 15:46:18.000511 dv-cinder-835384-9-iscsi devstack@keystone.service[80143]: *** Starting uWSGI 2.0.18-debian (64bit) on [Thu Jun 23 15:46:17 2022] *** Jun 23 15:46:18.000511 dv-cinder-835384-9-iscsi devstack@keystone.service[80143]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jun 23 15:46:18.000511 dv-cinder-835384-9-iscsi devstack@keystone.service[80143]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jun 23 15:46:18.000511 dv-cinder-835384-9-iscsi devstack@keystone.service[80143]: nodename: dv-cinder-835384-9-iscsi Jun 23 15:46:18.000511 dv-cinder-835384-9-iscsi devstack@keystone.service[80143]: machine: x86_64 Jun 23 15:46:18.000511 dv-cinder-835384-9-iscsi devstack@keystone.service[80143]: clock source: unix Jun 23 15:46:18.000511 dv-cinder-835384-9-iscsi devstack@keystone.service[80143]: pcre jit disabled Jun 23 15:46:18.000511 dv-cinder-835384-9-iscsi devstack@keystone.service[80143]: detected number of CPU cores: 4 Jun 23 15:46:18.000511 dv-cinder-835384-9-iscsi devstack@keystone.service[80143]: current working directory: / Jun 23 15:46:18.000511 dv-cinder-835384-9-iscsi devstack@keystone.service[80143]: detected binary path: /usr/bin/uwsgi-core Jun 23 15:46:18.000511 dv-cinder-835384-9-iscsi devstack@keystone.service[80143]: your processes number limit is 47680 Jun 23 15:46:18.000511 dv-cinder-835384-9-iscsi devstack@keystone.service[80143]: your memory page size is 4096 bytes Jun 23 15:46:18.000511 dv-cinder-835384-9-iscsi devstack@keystone.service[80143]: detected max file descriptor number: 2048 Jun 23 15:46:18.000511 dv-cinder-835384-9-iscsi devstack@keystone.service[80143]: lock engine: pthread robust mutexes Jun 23 15:46:18.001369 dv-cinder-835384-9-iscsi devstack@keystone.service[80143]: thunder lock: enabled Jun 23 15:46:18.001369 dv-cinder-835384-9-iscsi devstack@keystone.service[80143]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jun 23 15:46:18.001369 dv-cinder-835384-9-iscsi devstack@keystone.service[80143]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] Jun 23 15:46:18.045515 dv-cinder-835384-9-iscsi devstack@keystone.service[80143]: Python main interpreter initialized at 0x5621ffe269c0 Jun 23 15:46:18.045515 dv-cinder-835384-9-iscsi devstack@keystone.service[80143]: python threads support enabled Jun 23 15:46:18.045515 dv-cinder-835384-9-iscsi devstack@keystone.service[80143]: your server socket listen backlog is limited to 100 connections Jun 23 15:46:18.045515 dv-cinder-835384-9-iscsi devstack@keystone.service[80143]: your mercy for graceful operations on workers is 90 seconds Jun 23 15:46:18.045894 dv-cinder-835384-9-iscsi devstack@keystone.service[80143]: mapped 403077 bytes (393 KB) for 2 cores Jun 23 15:46:18.045975 dv-cinder-835384-9-iscsi devstack@keystone.service[80143]: *** Operational MODE: preforking *** Jun 23 15:46:18.046233 dv-cinder-835384-9-iscsi devstack@keystone.service[80143]: *** uWSGI is running in multiple interpreter mode *** Jun 23 15:46:18.046233 dv-cinder-835384-9-iscsi devstack@keystone.service[80143]: spawned uWSGI master process (pid: 80143) Jun 23 15:46:18.046446 dv-cinder-835384-9-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jun 23 15:46:18.047558 dv-cinder-835384-9-iscsi devstack@keystone.service[80143]: spawned uWSGI worker 1 (pid: 80153, cores: 1) Jun 23 15:46:18.047558 dv-cinder-835384-9-iscsi devstack@keystone.service[80143]: spawned uWSGI worker 2 (pid: 80154, cores: 1) Jun 23 15:46:18.047558 dv-cinder-835384-9-iscsi devstack@keystone.service[80143]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jun 23 15:46:23.597154 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 23 15:46:23.597154 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: from cryptography.utils import int_from_bytes Jun 23 15:46:23.597895 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 23 15:46:23.597895 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: from cryptography.utils import int_from_bytes Jun 23 15:46:23.598025 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 23 15:46:23.598025 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: from cryptography.utils import int_from_bytes Jun 23 15:46:23.598447 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 23 15:46:23.598447 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: from cryptography.utils import int_from_bytes Jun 23 15:46:23.647425 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jun 23 15:46:23.647607 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jun 23 15:46:23.647721 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jun 23 15:46:23.647865 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] command line args: [] {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jun 23 15:46:23.647947 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jun 23 15:46:23.648034 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jun 23 15:46:23.648107 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] command line args: [] {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jun 23 15:46:23.648228 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] ================================================================================ {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jun 23 15:46:23.648360 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jun 23 15:46:23.648486 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] admin_token = **** {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 15:46:23.648569 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] ================================================================================ {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jun 23 15:46:23.648739 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] config_dir = [] {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 15:46:23.648816 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] admin_token = **** {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 15:46:23.648931 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 15:46:23.649034 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] config_dir = [] {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 15:46:23.649115 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] config_source = [] {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 15:46:23.649228 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 15:46:23.649312 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] debug = True {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 15:46:23.649415 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] config_source = [] {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 15:46:23.649490 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 15:46:23.649731 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] debug = True {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 15:46:23.649802 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 15:46:23.649886 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 15:46:23.650012 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 15:46:23.650090 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 15:46:23.650207 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 15:46:23.650291 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 15:46:23.650464 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 15:46:23.650542 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 15:46:23.650741 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 15:46:23.650830 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] list_limit = None {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 15:46:23.650940 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] log_config_append = None {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 15:46:23.651020 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] list_limit = None {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 15:46:23.651118 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 15:46:23.651222 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] log_config_append = None {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 15:46:23.651322 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] log_dir = None {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 15:46:23.651414 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 15:46:23.651515 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] log_file = None {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 15:46:23.651643 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] log_dir = None {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 15:46:23.651741 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] log_file = None {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 15:46:23.651848 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 15:46:23.651964 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 15:46:23.652039 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 15:46:23.652156 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 15:46:23.652237 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 15:46:23.652662 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 15:46:23.652748 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 15:46:23.652910 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 15:46:23.652992 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 15:46:23.653094 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 15:46:23.653171 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 15:46:23.653297 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 15:46:23.653371 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 15:46:23.653520 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 15:46:23.653604 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 15:46:23.653775 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 15:46:23.653858 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 15:46:23.654015 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 15:46:23.654131 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 15:46:23.654251 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 15:46:23.654326 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 15:46:23.654487 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 15:46:23.654602 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 15:46:23.654759 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 15:46:23.654846 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 15:46:23.655012 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 15:46:23.655092 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 15:46:23.655262 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 15:46:23.655336 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 15:46:23.655542 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 15:46:23.655775 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.46/identity {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 15:46:23.655962 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] publish_errors = False {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 15:46:23.656086 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.46/identity {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 15:46:23.656199 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 15:46:23.656274 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] publish_errors = False {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 15:46:23.656433 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 15:46:23.656556 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 15:46:23.656638 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 15:46:23.656750 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 15:46:23.656828 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 15:46:23.656940 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 15:46:23.657074 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 15:46:23.657157 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 15:46:23.657312 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 15:46:23.657392 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] standard_threads = False {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 15:46:23.657529 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 15:46:23.657603 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] standard_threads = False {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 15:46:23.657777 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 15:46:23.657853 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 15:46:23.657955 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 15:46:23.658029 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 15:46:23.658142 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] use_journal = False {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 15:46:23.658224 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 15:46:23.658412 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] use_json = False {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 15:46:23.658496 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] use_journal = False {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 15:46:23.658572 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] use_stderr = False {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 15:46:23.658649 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] use_json = False {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 15:46:23.658754 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] use_syslog = False {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 15:46:23.658827 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] use_stderr = False {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 15:46:23.658902 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] use_syslog = False {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 15:46:23.658970 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 15:46:23.659104 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 23 15:46:23.659181 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.659366 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.659448 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.659617 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.659698 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.659867 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.659947 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.660115 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.660196 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.660356 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.660430 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.660601 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.660682 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.660836 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.660911 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.661083 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.661163 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.661331 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.661407 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.661581 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.661656 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.661815 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.661895 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.662056 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.662196 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.662279 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.662354 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.662526 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.662608 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.662798 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.662876 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.663027 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.663109 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.663280 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.663356 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.663537 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.663611 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.663783 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.663859 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.664022 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.664098 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.664262 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.664337 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.664520 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.664596 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.664774 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.664849 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.665026 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.665100 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.665261 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.665338 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.665497 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.665572 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.665728 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.665810 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.665962 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.666036 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.666203 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.666278 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.666448 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.666534 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.666682 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.666786 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.666936 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.667012 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.667192 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.667283 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.667444 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.667519 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.667680 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.667753 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.667923 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.668005 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.668154 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.668228 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.668390 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.668466 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.668627 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.668780 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.668853 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.668927 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.669078 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.669153 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.669310 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.669391 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.669540 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.669614 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.669774 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.669847 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.670014 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.670097 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.670244 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.670324 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.670481 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.670557 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.670733 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.670825 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.670973 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.671053 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.671210 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.671280 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.671448 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.671521 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.671684 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.671754 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.671921 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.671987 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.672159 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.672240 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.672397 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.672467 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.672631 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.672705 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.672867 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.672948 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.673098 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.673171 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.673330 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.673403 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.673572 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.673639 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.673810 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.673877 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.674053 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.674289 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.674552 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.674813 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.675051 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.675294 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.675535 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.675708 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.675793 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.675980 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.676054 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.676225 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.676303 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.676465 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.676540 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.676712 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.676800 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.676951 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.677027 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.677190 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.677280 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.677444 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.677528 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.677687 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.677763 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.677928 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.678003 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.678165 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.678247 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.678403 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.678479 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.678641 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.678770 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.678905 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.678992 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.679146 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.679228 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.679384 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.679460 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.679641 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.679725 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.679888 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.679963 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.680126 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.680200 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.680363 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.680450 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.680601 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.680686 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.680840 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.680914 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.681080 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.681227 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.681306 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.681375 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.681541 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.681617 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.681777 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.681861 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.682013 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.682101 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.682252 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.682328 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.682487 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.682561 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.683239 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.683339 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.683492 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.683574 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.683726 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.683810 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.683959 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.684044 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.684198 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.684303 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.684431 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.684568 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.684673 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.684815 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.684899 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.685053 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] auth.external = None {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.685149 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.685289 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.685373 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.685589 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.685673 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.685827 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.685904 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.686064 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] auth.password = None {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.686153 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.686299 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] auth.token = None {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.686392 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] auth.external = None {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.686544 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.686626 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.686810 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.686900 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.687064 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.687149 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.687302 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.687378 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] auth.password = None {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.687537 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] auth.token = None {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.687613 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.687781 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.687865 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.688018 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.688093 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.688259 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] credential.caching = True {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.688335 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.688490 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.688577 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.688739 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.688817 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.688988 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.689064 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.689222 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.689306 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.689460 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] credential.caching = True {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.689536 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.689696 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.689772 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.689946 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.690030 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.690195 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.690272 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.690433 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.690510 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.690679 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.690779 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.690938 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.691015 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.691212 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.691289 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.691468 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.691553 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.691707 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.691784 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.691962 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.692045 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.692205 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] federation.caching = True {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.692289 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.692451 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.692527 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.692714 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.692790 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.692963 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.693046 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.693199 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.693276 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.693445 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] federation.caching = True {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.693520 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.693684 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.693769 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.693932 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.694008 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.694173 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.694247 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.694404 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.694488 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.694652 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.694758 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.694927 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.695005 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.695168 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] identity.caching = True {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.695256 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.695413 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.695489 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.695665 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.695740 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.695908 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.695988 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.696150 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.696227 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.696388 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] identity.caching = True {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.696463 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.696628 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.696706 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.696870 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.696946 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.697112 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.697188 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.697349 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.697433 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.697593 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.697674 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.697825 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.697901 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.698057 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.698134 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.698307 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.698390 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.699148 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.699229 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.699388 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.699468 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.699622 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.699700 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.699864 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.699940 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.700112 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.700190 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.700361 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.700438 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.700611 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.700688 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.700845 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.700922 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.701088 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.701166 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.701324 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.701400 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.701564 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.701642 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.701809 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.701886 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.702039 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.702120 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.702277 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.702355 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.702510 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.702587 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.702772 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.702852 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.703018 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.703097 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.703258 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.703334 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.703506 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.703581 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.703738 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.703822 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.703974 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.704051 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.704217 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.704294 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.704450 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.704534 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.704687 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.704771 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.704927 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.705003 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.705160 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.705237 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.705400 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.705475 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.705630 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.705705 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.705867 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.705941 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.706104 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.706179 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.706343 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.706419 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.706579 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.706658 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.706837 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.706914 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.707093 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.707169 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.707331 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.707409 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.707571 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.707648 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.707802 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.707878 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.708037 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] ldap.user = None {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.708126 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.708281 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.708358 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.708516 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.708600 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.708754 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.708831 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.708990 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.709066 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.709222 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] ldap.user = None {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.709309 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.709464 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.709541 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.709708 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.709783 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.709946 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.710030 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.710182 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.710258 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.710421 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.710499 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.710662 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.710760 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.710920 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.710996 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.711152 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.711229 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.711388 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.711473 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.711624 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.711702 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.711861 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.711937 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.712092 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.712178 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.712340 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.712417 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.712590 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.712666 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.712831 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.712920 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.713076 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.713152 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.713314 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.713386 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.713544 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.713619 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.713792 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.713868 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.714049 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.714129 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.714288 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.714373 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.714526 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.714602 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.714782 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.714864 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.715027 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.715135 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.715268 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.715351 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.715501 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.715577 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.715739 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.715823 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.715976 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.716051 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.716222 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.716299 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.716456 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.716540 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.716691 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.716766 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.716929 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.717006 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] resource.caching = True {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.717180 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.717263 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.717421 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.717497 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.717656 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.717732 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.717893 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.717977 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.718134 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] resource.caching = True {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.718210 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.718377 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.718453 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.718624 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.718722 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.718888 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.718965 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.719133 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.719208 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.719374 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] role.caching = True {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.719528 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.719623 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.719699 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] role.driver = sql {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.719862 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.719938 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.720098 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.720182 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.720340 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.720416 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.720579 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] role.caching = True {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.720655 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.720830 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] role.driver = sql {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.720913 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.721062 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.721139 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.721313 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.721390 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.721563 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.721641 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.721811 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.721886 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.722047 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.722122 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.722297 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.722374 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.722546 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.722622 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.722818 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.722897 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.723069 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.723154 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.723302 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.723476 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.723552 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.723721 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.723810 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.723979 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.724057 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.724236 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.724313 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.724491 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.724662 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.724743 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.724903 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.724978 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.725157 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.725234 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.725410 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.725488 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.725662 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.725744 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.725912 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.725988 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.726147 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.726224 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.726383 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.726459 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.726641 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.726788 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.726905 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.726981 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.727144 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.727221 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.727380 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.727465 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.727616 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.727693 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.727850 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.727925 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.728093 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] token.caching = True {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.728177 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.728335 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.728411 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.728585 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.728661 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.728823 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.728908 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.729056 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.729131 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.729293 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] token.caching = True {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.729370 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.729532 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.729616 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.729807 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.729885 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.730046 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.730124 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.730295 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.730380 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.730537 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.730615 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.730797 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.730877 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.731041 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.731130 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.731281 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.731357 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.731531 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.731606 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.731765 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.731850 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.732005 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.732082 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.732241 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.732318 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.732489 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.732574 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] database.connection = **** {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.732737 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.732831 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.732968 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.733045 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.733212 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.733289 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.733503 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.733588 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.733743 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.733819 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] database.connection = **** {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.733987 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.734072 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.734227 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.734304 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.734467 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.734546 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.734745 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.734827 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.734968 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.735046 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.735207 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.735287 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.735445 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.735521 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.735680 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.735756 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.735925 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.736003 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.736158 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.736234 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.736395 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.736470 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.736627 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.736704 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.736866 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.736942 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.737042 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80154) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jun 23 15:46:23.737120 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.737340 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.737573 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.737814 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.738050 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 23 15:46:23.738221 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80153) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jun 23 15:46:23.768937 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80154) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 15:46:23.769161 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80154) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 15:46:23.769388 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80154) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 15:46:23.775911 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80153) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 15:46:23.776149 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80153) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 15:46:23.776391 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80153) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 15:46:23.778437 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80154) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 15:46:23.785340 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80153) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 15:46:23.806450 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80154) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 15:46:23.806678 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80154) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 15:46:23.811790 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80153) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 15:46:23.812020 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80153) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 15:46:23.950356 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80154) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 15:46:23.951450 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80153) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 15:46:24.000286 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80154) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 15:46:24.000456 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80153) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 15:46:24.000551 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80154) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 15:46:24.000699 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80153) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 15:46:24.065414 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80153) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 15:46:24.065639 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80153) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 15:46:24.065858 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80154) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 15:46:24.065966 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80153) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 15:46:24.066078 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80153) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 15:46:24.066161 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80154) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 15:46:24.066302 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80153) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 15:46:24.066385 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80154) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 15:46:24.066524 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80153) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 15:46:24.066639 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80154) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 15:46:24.066768 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80154) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 15:46:24.066990 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80154) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 15:46:24.092767 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80153) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 15:46:24.092931 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80154) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 15:46:24.093021 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80153) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 15:46:24.093128 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80154) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 15:46:24.093212 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80153) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 15:46:24.093357 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80154) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 15:46:24.093439 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80153) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 15:46:24.093580 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80154) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 15:46:24.093662 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80153) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 15:46:24.093797 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80154) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 15:46:24.093879 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80153) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 15:46:24.094010 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80154) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 15:46:24.094092 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80153) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 15:46:24.094225 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80154) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 15:46:24.105151 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80153) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 15:46:24.105549 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80154) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 15:46:24.117772 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80153) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 15:46:24.118004 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80153) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 15:46:24.118105 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80154) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 15:46:24.118230 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80153) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 15:46:24.118314 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80154) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 15:46:24.118454 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80153) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 15:46:24.118536 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80154) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 15:46:24.118673 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80153) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 15:46:24.118784 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80154) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 15:46:24.118915 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80153) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 15:46:24.118999 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80154) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 15:46:24.119132 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80154) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 15:46:24.119213 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80153) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 15:46:24.119350 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80154) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 23 15:46:24.137026 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80153) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 15:46:24.137236 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80153) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 23 15:46:24.137465 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80154) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 15:46:24.137580 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80153) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 15:46:24.137679 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80154) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 23 15:46:24.137785 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80153) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 23 15:46:24.138007 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80154) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 15:46:24.138088 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80153) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 15:46:24.138208 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80154) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 23 15:46:24.138298 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80153) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 15:46:24.138485 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80154) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 15:46:24.138597 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80153) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 15:46:24.138772 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80154) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 15:46:24.138856 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80153) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 15:46:24.139036 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80154) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 15:46:24.139465 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80154) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 15:46:24.149706 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80153) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 15:46:24.149978 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80153) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 15:46:24.150205 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80154) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 15:46:24.150289 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80153) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 15:46:24.150485 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80154) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 15:46:24.150583 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80153) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 15:46:24.150781 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80154) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 15:46:24.151072 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80154) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 15:46:24.159719 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80153) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 15:46:24.160288 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80154) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 15:46:24.163054 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80153) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 15:46:24.163331 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80153) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 15:46:24.163610 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80153) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 15:46:24.163694 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80154) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 15:46:24.163892 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80153) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 15:46:24.163989 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80154) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 15:46:24.164169 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80153) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 15:46:24.164251 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80154) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 15:46:24.164424 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80153) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 15:46:24.164504 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80154) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 15:46:24.164690 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80153) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 15:46:24.164772 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80154) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 15:46:24.164962 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80154) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 15:46:24.165039 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80153) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 15:46:24.165230 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80153) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 15:46:24.165308 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80154) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 15:46:24.165503 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80153) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 15:46:24.165583 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80154) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 15:46:24.165769 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80153) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 15:46:24.165853 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80154) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 15:46:24.166042 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80154) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 15:46:24.166313 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80154) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 15:46:24.192255 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80153) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 15:46:24.193867 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80153) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 15:46:24.194142 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80153) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 15:46:24.197598 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80154) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 15:46:24.199980 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80153) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 15:46:24.200346 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80154) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 15:46:24.200482 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80153) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 15:46:24.200637 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80153) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 15:46:24.200797 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80154) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 15:46:24.206112 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80154) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 15:46:24.206511 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80154) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 15:46:24.206902 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80154) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 15:46:24.207955 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80153) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 15:46:24.208236 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80153) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 15:46:24.212730 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80153) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 15:46:24.213012 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80153) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 15:46:24.213295 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80153) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 15:46:24.213577 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80153) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 15:46:24.213857 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80153) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 15:46:24.214678 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80153) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 15:46:24.214796 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80154) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 15:46:24.215057 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80153) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 15:46:24.215150 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80154) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 15:46:24.215226 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80153) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 15:46:24.219589 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80154) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 15:46:24.219877 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80154) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 15:46:24.220166 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80154) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 15:46:24.220456 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80154) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 15:46:24.220740 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80154) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 15:46:24.221016 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80154) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 15:46:24.221290 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80154) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 15:46:24.221564 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80154) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 15:46:24.233943 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80153) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 15:46:24.234216 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80153) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 15:46:24.238854 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80153) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 15:46:24.240445 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80154) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 15:46:24.240723 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80154) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 15:46:24.242869 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80153) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 15:46:24.243264 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80153) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 15:46:24.245876 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80154) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 15:46:24.248831 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80153) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 15:46:24.249545 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80154) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 15:46:24.249948 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80154) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 15:46:24.252465 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80153) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 15:46:24.255026 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80154) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 15:46:24.256130 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80153) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 15:46:24.256432 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80153) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 15:46:24.256720 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80153) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 15:46:24.257006 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80153) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 15:46:24.257289 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80153) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 15:46:24.257573 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80153) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 15:46:24.258660 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80154) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 15:46:24.262938 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80154) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 15:46:24.263238 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80154) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 15:46:24.263529 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80154) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 15:46:24.263813 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80154) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 15:46:24.264100 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80154) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 15:46:24.264383 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80154) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 15:46:24.277188 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80153) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 15:46:24.277731 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80153) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 15:46:24.277996 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80153) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 15:46:24.278267 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80153) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 15:46:24.283742 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80154) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 15:46:24.284025 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80154) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 15:46:24.284294 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80154) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 15:46:24.284580 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80154) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 15:46:24.286173 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80153) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 15:46:24.287891 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80153) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 15:46:24.288168 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80153) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 15:46:24.291396 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80153) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 15:46:24.291674 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80153) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 15:46:24.291955 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80153) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 15:46:24.292261 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80153) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 15:46:24.292548 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80153) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 15:46:24.292630 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80154) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 15:46:24.294257 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80154) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 15:46:24.294546 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80154) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 15:46:24.297842 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80154) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 15:46:24.298141 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80154) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 15:46:24.298446 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80154) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 15:46:24.298780 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80154) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 15:46:24.299122 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80154) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 15:46:24.305802 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80153) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 15:46:24.306077 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80153) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 15:46:24.306368 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80153) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 15:46:24.306652 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80153) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 15:46:24.308862 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80153) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 15:46:24.309155 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80153) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 15:46:24.309445 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80153) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 15:46:24.312948 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80154) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 15:46:24.313235 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80154) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 15:46:24.313529 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80154) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 15:46:24.313813 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80154) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 15:46:24.315538 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80154) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 15:46:24.315835 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80154) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 15:46:24.316130 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80154) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 15:46:24.327908 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80153) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 15:46:24.331242 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80153) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 15:46:24.334540 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80153) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 15:46:24.334680 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80154) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 15:46:24.336322 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80153) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 15:46:24.337940 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80154) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 15:46:24.338071 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80153) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 15:46:24.338217 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80153) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 15:46:24.338493 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80153) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 15:46:24.341324 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80154) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 15:46:24.343120 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80154) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 15:46:24.344771 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80154) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 15:46:24.345127 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80154) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 15:46:24.345337 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80154) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 15:46:24.345786 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80153) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 15:46:24.347491 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80153) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 15:46:24.350824 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80153) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 15:46:24.351045 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80153) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 15:46:24.351434 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80153) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 15:46:24.351713 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80153) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 15:46:24.352689 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80154) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 15:46:24.354319 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80154) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 15:46:24.357710 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80154) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 15:46:24.358003 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80154) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 15:46:24.358393 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80154) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 15:46:24.358674 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80154) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 15:46:24.360819 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80153) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 15:46:24.361106 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80153) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 15:46:24.361386 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80153) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 15:46:24.367814 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80154) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 15:46:24.368101 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80154) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 15:46:24.368385 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80154) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 15:46:24.369257 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80153) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 15:46:24.369532 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80153) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 15:46:24.369809 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80153) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 15:46:24.370081 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80153) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 15:46:24.370907 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80153) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 15:46:24.371287 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80153) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 15:46:24.371572 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80153) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 15:46:24.371852 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80153) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 15:46:24.372127 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80153) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 15:46:24.372406 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80153) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 15:46:24.372681 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80153) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 15:46:24.372952 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80153) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 15:46:24.373220 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80153) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 15:46:24.373494 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80153) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 15:46:24.376890 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80154) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 23 15:46:24.377173 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80154) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 15:46:24.377456 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80154) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 15:46:24.377756 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80154) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 15:46:24.378014 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80154) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 15:46:24.378386 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80154) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 15:46:24.378675 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80154) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 15:46:24.378977 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80154) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 15:46:24.379256 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80154) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 15:46:24.379540 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80154) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 15:46:24.379826 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80154) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 15:46:24.380096 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80154) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 15:46:24.380378 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80154) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 15:46:24.380653 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80154) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 23 15:46:24.412393 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x5621ffe269c0 pid: 80153 (default app) Jun 23 15:46:24.415836 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7955a6ec-27e3-45be-8054-de74ee9acc8f None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:46:24.416137 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7955a6ec-27e3-45be-8054-de74ee9acc8f None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:46:24.416419 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7955a6ec-27e3-45be-8054-de74ee9acc8f None None] PATH_INFO: `/v3` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:46:24.419760 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x5621ffe269c0 pid: 80154 (default app) Jun 23 15:46:24.420647 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1/1] 10.222.0.46 () {50 vars in 791 bytes} [Thu Jun 23 15:46:24 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:46:32.153342 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4daa6574-13f4-42e2-847d-f3d5650c5f2f None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:46:32.153925 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4daa6574-13f4-42e2-847d-f3d5650c5f2f None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:46:32.153925 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4daa6574-13f4-42e2-847d-f3d5650c5f2f None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:46:32.158211 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1/2] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jun 23 15:46:32 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 15:46:32.163702 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7eb477-e6d6-4ca0-af18-562e64412d4b None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:46:32.164086 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7eb477-e6d6-4ca0-af18-562e64412d4b None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:46:32.164261 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7eb477-e6d6-4ca0-af18-562e64412d4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:46:32.234763 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: INFO dbcounter [None req-ce7eb477-e6d6-4ca0-af18-562e64412d4b None None] Registered counter for database keystone Jun 23 15:46:32.256809 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG oslo_db.sqlalchemy.engines [None req-ce7eb477-e6d6-4ca0-af18-562e64412d4b None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80153) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jun 23 15:46:32.263829 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writer thread running {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jun 23 15:46:32.446145 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG passlib.handlers.bcrypt [None req-ce7eb477-e6d6-4ca0-af18-562e64412d4b None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80153) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jun 23 15:46:32.446487 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG passlib.handlers.bcrypt [None req-ce7eb477-e6d6-4ca0-af18-562e64412d4b None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80153) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jun 23 15:46:32.505136 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-ce7eb477-e6d6-4ca0-af18-562e64412d4b None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:46:32.506790 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-ce7eb477-e6d6-4ca0-af18-562e64412d4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:46:32.630632 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 2/3] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jun 23 15:46:32 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 468 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Jun 23 15:46:32.766415 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4daa6574-13f4-42e2-847d-f3d5650c5f2f None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:46:32.836562 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: INFO dbcounter [None req-4daa6574-13f4-42e2-847d-f3d5650c5f2f None None] Registered counter for database keystone Jun 23 15:46:32.856871 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG oslo_db.sqlalchemy.engines [None req-4daa6574-13f4-42e2-847d-f3d5650c5f2f None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80154) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jun 23 15:46:32.863200 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writer thread running {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jun 23 15:46:32.981897 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4daa6574-13f4-42e2-847d-f3d5650c5f2f None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:46:32.986175 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79792b5a-03d6-4daf-bd09-fbfd1565e98e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:46:32.986867 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79792b5a-03d6-4daf-bd09-fbfd1565e98e admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:46:32.987153 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79792b5a-03d6-4daf-bd09-fbfd1565e98e admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:46:32.987428 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79792b5a-03d6-4daf-bd09-fbfd1565e98e admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:46:32.995602 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79792b5a-03d6-4daf-bd09-fbfd1565e98e admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:46:33.047398 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: WARNING keystone.common.rbac_enforcer.enforcer [None req-79792b5a-03d6-4daf-bd09-fbfd1565e98e admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jun 23 15:46:33.047668 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79792b5a-03d6-4daf-bd09-fbfd1565e98e admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:46:33.057539 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 2/4] 10.222.0.46 () {58 vars in 1324 bytes} [Thu Jun 23 15:46:32 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 23 15:46:33.063612 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7eb477-e6d6-4ca0-af18-562e64412d4b None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:46:33.078423 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7eb477-e6d6-4ca0-af18-562e64412d4b None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:46:33.082814 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991fc66d-4a3a-4c88-9abd-1a4ffc910ab2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:46:33.083934 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991fc66d-4a3a-4c88-9abd-1a4ffc910ab2 admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:46:33.084220 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991fc66d-4a3a-4c88-9abd-1a4ffc910ab2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:46:33.084493 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991fc66d-4a3a-4c88-9abd-1a4ffc910ab2 admin admin] PATH_INFO: `/v3/projects/110dee34db4740f989f715e757ee809c` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:46:33.089101 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991fc66d-4a3a-4c88-9abd-1a4ffc910ab2 admin admin] RBAC: Authorizing `identity:get_project(project_id=110dee34db4740f989f715e757ee809c)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:46:33.138734 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: WARNING keystone.common.rbac_enforcer.enforcer [None req-991fc66d-4a3a-4c88-9abd-1a4ffc910ab2 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jun 23 15:46:33.139074 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991fc66d-4a3a-4c88-9abd-1a4ffc910ab2 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:46:33.143640 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 3/5] 10.222.0.46 () {56 vars in 1207 bytes} [Thu Jun 23 15:46:33 2022] GET /identity/v3/projects/110dee34db4740f989f715e757ee809c => generated 344 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:46:35.184107 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e38d516-51e5-4566-95a2-0babbc5c6623 None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:46:35.184668 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e38d516-51e5-4566-95a2-0babbc5c6623 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:46:35.184668 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e38d516-51e5-4566-95a2-0babbc5c6623 None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:46:35.188963 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 3/6] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jun 23 15:46:35 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 15:46:35.194831 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baebc52a-ec6d-4903-a2b9-6a9637298ed4 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:46:35.195312 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baebc52a-ec6d-4903-a2b9-6a9637298ed4 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:46:35.195430 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baebc52a-ec6d-4903-a2b9-6a9637298ed4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:46:35.215156 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-baebc52a-ec6d-4903-a2b9-6a9637298ed4 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:46:35.216693 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-baebc52a-ec6d-4903-a2b9-6a9637298ed4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:46:35.224772 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 4/7] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jun 23 15:46:35 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Jun 23 15:46:35.247744 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e38d516-51e5-4566-95a2-0babbc5c6623 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:46:35.260117 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e38d516-51e5-4566-95a2-0babbc5c6623 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:46:35.265274 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28969467-75ef-4238-ad21-5d31b0db9f30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:46:35.265821 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28969467-75ef-4238-ad21-5d31b0db9f30 admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:46:35.266108 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28969467-75ef-4238-ad21-5d31b0db9f30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:46:35.266391 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28969467-75ef-4238-ad21-5d31b0db9f30 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:46:35.274569 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28969467-75ef-4238-ad21-5d31b0db9f30 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:46:35.277304 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28969467-75ef-4238-ad21-5d31b0db9f30 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:46:35.286413 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 4/8] 10.222.0.46 () {58 vars in 1324 bytes} [Thu Jun 23 15:46:35 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 23 15:46:35.292204 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baebc52a-ec6d-4903-a2b9-6a9637298ed4 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:46:35.301591 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baebc52a-ec6d-4903-a2b9-6a9637298ed4 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:46:35.306099 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd914438-7f80-4102-9aa4-437502a2bd48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:46:35.307012 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd914438-7f80-4102-9aa4-437502a2bd48 admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:46:35.307302 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd914438-7f80-4102-9aa4-437502a2bd48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:46:35.307582 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd914438-7f80-4102-9aa4-437502a2bd48 admin admin] PATH_INFO: `/v3/users/e9c7fd1e1ef54284a2b594ce3d58fc5a` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:46:35.309243 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd914438-7f80-4102-9aa4-437502a2bd48 admin admin] RBAC: Authorizing `identity:get_user(user_id=e9c7fd1e1ef54284a2b594ce3d58fc5a)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:46:35.311280 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd914438-7f80-4102-9aa4-437502a2bd48 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:46:35.312979 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 5/9] 10.222.0.46 () {56 vars in 1198 bytes} [Thu Jun 23 15:46:35 2022] GET /identity/v3/users/e9c7fd1e1ef54284a2b594ce3d58fc5a => generated 247 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:46:37.397519 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a07dbae-a502-46f5-ac37-c270bccf1672 None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:46:37.398057 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a07dbae-a502-46f5-ac37-c270bccf1672 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:46:37.398057 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a07dbae-a502-46f5-ac37-c270bccf1672 None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:46:37.399637 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 5/10] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jun 23 15:46:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 15:46:37.405277 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556337b8-91b7-4641-8eee-2bfefbf415c7 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:46:37.405739 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556337b8-91b7-4641-8eee-2bfefbf415c7 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:46:37.406002 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556337b8-91b7-4641-8eee-2bfefbf415c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:46:37.423851 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-556337b8-91b7-4641-8eee-2bfefbf415c7 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:46:37.425084 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-556337b8-91b7-4641-8eee-2bfefbf415c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:46:37.477422 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b787a86-1c02-49a0-9fa0-d679ba56f23a None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:46:37.478433 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b787a86-1c02-49a0-9fa0-d679ba56f23a None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:46:37.478433 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b787a86-1c02-49a0-9fa0-d679ba56f23a None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:46:37.479994 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 6/11] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jun 23 15:46:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 15:46:37.485695 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d54e63-6eab-4885-9942-9c94f9103673 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:46:37.486111 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d54e63-6eab-4885-9942-9c94f9103673 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:46:37.486278 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d54e63-6eab-4885-9942-9c94f9103673 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:46:37.514946 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 6/12] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jun 23 15:46:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 15:46:37.527405 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG passlib.handlers.bcrypt [None req-74d54e63-6eab-4885-9942-9c94f9103673 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80154) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jun 23 15:46:37.527854 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG passlib.handlers.bcrypt [None req-74d54e63-6eab-4885-9942-9c94f9103673 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80154) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jun 23 15:46:37.539163 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556337b8-91b7-4641-8eee-2bfefbf415c7 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:46:37.565047 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:46:37.574418 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-74d54e63-6eab-4885-9942-9c94f9103673 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:46:37.576186 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-74d54e63-6eab-4885-9942-9c94f9103673 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:46:37.649289 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556337b8-91b7-4641-8eee-2bfefbf415c7 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:46:37.676215 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 7/13] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jun 23 15:46:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 15:46:37.684840 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a384fb3-0f78-48ad-9105-92a235d6de28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:46:37.685773 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a384fb3-0f78-48ad-9105-92a235d6de28 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:46:37.685937 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a384fb3-0f78-48ad-9105-92a235d6de28 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:46:37.686222 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a384fb3-0f78-48ad-9105-92a235d6de28 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:46:37.698413 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a384fb3-0f78-48ad-9105-92a235d6de28 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 15:46:37.698734 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d54e63-6eab-4885-9942-9c94f9103673 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:46:37.710829 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-8a384fb3-0f78-48ad-9105-92a235d6de28 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 15:46:37.710829 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:46:37.710829 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:46:37.710829 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:46:37.710829 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:46:37.710829 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:46:37.710829 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:46:37.710829 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:46:37.710829 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:46:37.710829 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:46:37.710829 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:46:37.710829 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:46:37.710829 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 15:46:37.710829 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 15:46:37.710829 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 15:46:37.710829 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 15:46:37.711959 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 15:46:37.711959 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 15:46:37.711959 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 15:46:37.711959 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 15:46:37.711959 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:46:37.711959 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:46:37.711959 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:46:37.711959 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:46:37.711959 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:46:37.711959 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:46:37.711959 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:46:37.711959 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Jun 23 15:46:37.711959 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:46:37.711959 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:46:37.711959 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:46:37.711959 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:46:37.712844 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:46:37.712844 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:46:37.712844 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:46:37.712844 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:46:37.712844 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 15:46:37.712844 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 15:46:37.712844 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 15:46:37.712844 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 15:46:37.712844 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 15:46:37.712844 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 15:46:37.712844 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 15:46:37.712844 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Jun 23 15:46:37.718716 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 7/14] 10.222.0.46 () {56 vars in 1096 bytes} [Thu Jun 23 15:46:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 15:46:37.719145 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a384fb3-0f78-48ad-9105-92a235d6de28 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:46:37.804975 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d54e63-6eab-4885-9942-9c94f9103673 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:46:37.807121 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:46:37.807331 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:46:37.828234 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a384fb3-0f78-48ad-9105-92a235d6de28 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:46:37.844690 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0c7aae-dd6b-42ee-a3cf-f38036900d57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:46:37.846037 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0c7aae-dd6b-42ee-a3cf-f38036900d57 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:46:37.846545 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0c7aae-dd6b-42ee-a3cf-f38036900d57 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:46:37.846958 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0c7aae-dd6b-42ee-a3cf-f38036900d57 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:46:37.867293 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9022ddf-53f2-4f18-9bbb-bd8acb2b39ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:46:37.867976 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9022ddf-53f2-4f18-9bbb-bd8acb2b39ca None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:46:37.868258 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9022ddf-53f2-4f18-9bbb-bd8acb2b39ca None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:46:37.868690 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9022ddf-53f2-4f18-9bbb-bd8acb2b39ca None admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:46:37.869483 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9022ddf-53f2-4f18-9bbb-bd8acb2b39ca None admin] RBAC: Adding query filter params (name=admin) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:46:37.869817 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9022ddf-53f2-4f18-9bbb-bd8acb2b39ca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:46:37.871446 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9022ddf-53f2-4f18-9bbb-bd8acb2b39ca None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:46:37.884119 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 8/15] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Jun 23 15:46:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:46:37.889608 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9022ddf-53f2-4f18-9bbb-bd8acb2b39ca None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:46:37.924085 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0c7aae-dd6b-42ee-a3cf-f38036900d57 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:46:37.925528 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0c7aae-dd6b-42ee-a3cf-f38036900d57 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:46:37.992193 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9022ddf-53f2-4f18-9bbb-bd8acb2b39ca None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:46:38.027474 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 8/16] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Jun 23 15:46:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 23 15:46:38.028059 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b562143f-2b8b-4c89-be45-89e4e4854bf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:46:38.028962 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b562143f-2b8b-4c89-be45-89e4e4854bf6 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:46:38.029478 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b562143f-2b8b-4c89-be45-89e4e4854bf6 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:46:38.029773 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b562143f-2b8b-4c89-be45-89e4e4854bf6 None admin] PATH_INFO: `/v3/users/e9c7fd1e1ef54284a2b594ce3d58fc5a` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:46:38.031318 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b562143f-2b8b-4c89-be45-89e4e4854bf6 None admin] RBAC: Authorizing `identity:get_user(user_id=e9c7fd1e1ef54284a2b594ce3d58fc5a)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:46:38.032820 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b562143f-2b8b-4c89-be45-89e4e4854bf6 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:46:38.033392 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0c7aae-dd6b-42ee-a3cf-f38036900d57 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:46:38.034482 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 9/17] 10.222.0.46 () {56 vars in 1177 bytes} [Thu Jun 23 15:46:37 2022] GET /identity/v3/users/e9c7fd1e1ef54284a2b594ce3d58fc5a => generated 247 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 15:46:38.037000 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:46:38.039986 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b562143f-2b8b-4c89-be45-89e4e4854bf6 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:46:38.056180 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:46:38.131876 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0c7aae-dd6b-42ee-a3cf-f38036900d57 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:46:38.142340 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b562143f-2b8b-4c89-be45-89e4e4854bf6 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:46:38.166913 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6c7569-4334-4a7a-b889-3cf73401e95d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:46:38.167724 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6c7569-4334-4a7a-b889-3cf73401e95d None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:46:38.168009 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6c7569-4334-4a7a-b889-3cf73401e95d None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:46:38.168289 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6c7569-4334-4a7a-b889-3cf73401e95d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:46:38.177471 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bf1ba0-0f95-4fe6-b27a-929375050ea7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:46:38.178210 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bf1ba0-0f95-4fe6-b27a-929375050ea7 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:46:38.178491 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bf1ba0-0f95-4fe6-b27a-929375050ea7 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:46:38.178798 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bf1ba0-0f95-4fe6-b27a-929375050ea7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:46:38.179990 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7bf1ba0-0f95-4fe6-b27a-929375050ea7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:46:38.181460 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7bf1ba0-0f95-4fe6-b27a-929375050ea7 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:46:38.183161 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 10/18] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Jun 23 15:46:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 15:46:38.189126 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bf1ba0-0f95-4fe6-b27a-929375050ea7 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:46:38.189379 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6c7569-4334-4a7a-b889-3cf73401e95d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:46:38.190948 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6c7569-4334-4a7a-b889-3cf73401e95d None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:46:38.214450 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-0f6c7569-4334-4a7a-b889-3cf73401e95d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 15:46:38.214450 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:46:38.214450 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:46:38.214450 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:46:38.214450 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:46:38.214450 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:46:38.214450 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:46:38.214450 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:46:38.214450 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:46:38.214450 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:46:38.214450 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:46:38.214450 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:46:38.214450 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 15:46:38.214450 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 15:46:38.214450 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 15:46:38.215567 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 15:46:38.215567 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:46:38.215567 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:46:38.215567 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:46:38.215567 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:46:38.215567 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:46:38.215567 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:46:38.215567 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:46:38.215567 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Jun 23 15:46:38.215567 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:46:38.215567 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:46:38.215567 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:46:38.215567 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:46:38.215567 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:46:38.215567 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:46:38.215567 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:46:38.215567 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:46:38.216460 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 15:46:38.216460 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 15:46:38.216460 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 15:46:38.216460 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Jun 23 15:46:38.216460 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 9/19] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Jun 23 15:46:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:46:38.221380 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6c7569-4334-4a7a-b889-3cf73401e95d None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:46:38.272467 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:46:38.279316 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:46:38.293535 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bf1ba0-0f95-4fe6-b27a-929375050ea7 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:46:38.325712 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6c7569-4334-4a7a-b889-3cf73401e95d None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:46:38.327944 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec63150-243a-4b42-ae42-a82e5503d28d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:46:38.328616 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec63150-243a-4b42-ae42-a82e5503d28d None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:46:38.328897 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec63150-243a-4b42-ae42-a82e5503d28d None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:46:38.329170 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec63150-243a-4b42-ae42-a82e5503d28d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:46:38.330004 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec63150-243a-4b42-ae42-a82e5503d28d None admin] RBAC: Adding query filter params (role.id=812d3c567c52469cb2b35709fbdc857c, scope.domain.id=default, user.id=e9c7fd1e1ef54284a2b594ce3d58fc5a) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:46:38.330314 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec63150-243a-4b42-ae42-a82e5503d28d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:46:38.331813 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec63150-243a-4b42-ae42-a82e5503d28d None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:46:38.339567 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 11/20] 10.222.0.46 () {56 vars in 1322 bytes} [Thu Jun 23 15:46:38 2022] GET /identity/v3/role_assignments?user.id=e9c7fd1e1ef54284a2b594ce3d58fc5a&scope.domain.id=default&role.id=812d3c567c52469cb2b35709fbdc857c => generated 233 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 15:46:38.363526 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4e1359-679e-4b28-8355-69f3ea05917d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:46:38.364229 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4e1359-679e-4b28-8355-69f3ea05917d None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:46:38.364519 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4e1359-679e-4b28-8355-69f3ea05917d None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:46:38.364800 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4e1359-679e-4b28-8355-69f3ea05917d None admin] PATH_INFO: `/v3/domains` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:46:38.365564 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4e1359-679e-4b28-8355-69f3ea05917d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:46:38.365877 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4e1359-679e-4b28-8355-69f3ea05917d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:46:38.367380 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4e1359-679e-4b28-8355-69f3ea05917d None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:46:38.388239 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 10/21] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Jun 23 15:46:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 15:46:40.628813 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348cd808-a7a8-434e-aa0f-5bd8fdfb9cf5 None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:46:40.629320 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348cd808-a7a8-434e-aa0f-5bd8fdfb9cf5 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:46:40.629780 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348cd808-a7a8-434e-aa0f-5bd8fdfb9cf5 None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:46:40.631973 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 12/22] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jun 23 15:46:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 15:46:40.640400 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d46e3aa-a2c8-4e90-842a-d90db0fab61b None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:46:40.641336 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d46e3aa-a2c8-4e90-842a-d90db0fab61b None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:46:40.641823 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d46e3aa-a2c8-4e90-842a-d90db0fab61b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:46:40.660219 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-1d46e3aa-a2c8-4e90-842a-d90db0fab61b None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:46:40.661793 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-1d46e3aa-a2c8-4e90-842a-d90db0fab61b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:46:40.755135 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 11/23] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jun 23 15:46:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 15:46:40.794282 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348cd808-a7a8-434e-aa0f-5bd8fdfb9cf5 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:46:40.926347 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348cd808-a7a8-434e-aa0f-5bd8fdfb9cf5 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:46:40.974066 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e040b18-5224-4040-b732-81567a3054c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:46:40.974901 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e040b18-5224-4040-b732-81567a3054c3 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:46:40.975186 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e040b18-5224-4040-b732-81567a3054c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:46:40.975463 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e040b18-5224-4040-b732-81567a3054c3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:46:40.993845 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e040b18-5224-4040-b732-81567a3054c3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 15:46:41.002031 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-5e040b18-5224-4040-b732-81567a3054c3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 15:46:41.002031 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:46:41.002031 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:46:41.002031 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:46:41.002031 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:46:41.002031 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:46:41.002031 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:46:41.002031 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:46:41.002031 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:46:41.002031 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:46:41.002031 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:46:41.002031 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:46:41.002031 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 15:46:41.002031 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 15:46:41.002031 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 15:46:41.002031 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 15:46:41.003330 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 15:46:41.003330 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 15:46:41.003330 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 15:46:41.003330 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 15:46:41.003330 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:46:41.003330 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:46:41.003330 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:46:41.003330 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:46:41.003330 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:46:41.003330 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:46:41.003330 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:46:41.003330 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Jun 23 15:46:41.003330 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:46:41.003330 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:46:41.003330 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:46:41.003330 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:46:41.004479 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:46:41.004479 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:46:41.004479 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:46:41.004479 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:46:41.004479 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 15:46:41.004479 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 15:46:41.004479 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 15:46:41.004479 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 15:46:41.004479 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 15:46:41.004479 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 15:46:41.004479 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 15:46:41.004479 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Jun 23 15:46:41.004479 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 13/24] 10.222.0.46 () {56 vars in 1096 bytes} [Thu Jun 23 15:46:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:46:41.012334 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d46e3aa-a2c8-4e90-842a-d90db0fab61b None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:46:41.067290 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:46:41.135784 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d46e3aa-a2c8-4e90-842a-d90db0fab61b None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:46:41.152492 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4cabb8-fab7-4742-8833-d363b9436313 None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:46:41.152805 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4cabb8-fab7-4742-8833-d363b9436313 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:46:41.153050 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4cabb8-fab7-4742-8833-d363b9436313 None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:46:41.155208 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 14/25] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jun 23 15:46:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 15:46:41.160081 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6547c290-bb56-495b-bde7-475101e0b59e None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:46:41.160366 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6547c290-bb56-495b-bde7-475101e0b59e None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:46:41.160643 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6547c290-bb56-495b-bde7-475101e0b59e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:46:41.175181 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:46:41.189268 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-6547c290-bb56-495b-bde7-475101e0b59e None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:46:41.190718 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-6547c290-bb56-495b-bde7-475101e0b59e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:46:41.200246 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692f74bc-a94d-45e1-8772-ddfd90b7a096 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:46:41.200951 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692f74bc-a94d-45e1-8772-ddfd90b7a096 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:46:41.201243 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692f74bc-a94d-45e1-8772-ddfd90b7a096 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:46:41.201527 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692f74bc-a94d-45e1-8772-ddfd90b7a096 None admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:46:41.202306 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692f74bc-a94d-45e1-8772-ddfd90b7a096 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:46:41.202623 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692f74bc-a94d-45e1-8772-ddfd90b7a096 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:46:41.204526 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692f74bc-a94d-45e1-8772-ddfd90b7a096 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:46:41.225496 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 12/26] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Jun 23 15:46:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:46:41.231622 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692f74bc-a94d-45e1-8772-ddfd90b7a096 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:46:41.358453 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692f74bc-a94d-45e1-8772-ddfd90b7a096 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:46:41.368908 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:46:41.394331 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 15/27] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jun 23 15:46:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 15:46:41.395659 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2f1e83-6134-453e-905d-8afff5c26de7 None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:46:41.396002 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2f1e83-6134-453e-905d-8afff5c26de7 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:46:41.396287 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2f1e83-6134-453e-905d-8afff5c26de7 None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:46:41.397603 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 16/28] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jun 23 15:46:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 15:46:41.404349 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861d7cda-30a5-4f2d-b63f-76d4203769a8 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:46:41.404627 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861d7cda-30a5-4f2d-b63f-76d4203769a8 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:46:41.404852 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861d7cda-30a5-4f2d-b63f-76d4203769a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:46:41.412872 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a49344-f8bb-48c9-bb31-88ba60a43b99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:46:41.413786 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a49344-f8bb-48c9-bb31-88ba60a43b99 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:46:41.414070 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a49344-f8bb-48c9-bb31-88ba60a43b99 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:46:41.414345 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a49344-f8bb-48c9-bb31-88ba60a43b99 None admin] PATH_INFO: `/v3/users/e9c7fd1e1ef54284a2b594ce3d58fc5a` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:46:41.416016 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a49344-f8bb-48c9-bb31-88ba60a43b99 None admin] RBAC: Authorizing `identity:get_user(user_id=e9c7fd1e1ef54284a2b594ce3d58fc5a)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:46:41.417582 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a49344-f8bb-48c9-bb31-88ba60a43b99 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:46:41.419604 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 13/29] 10.222.0.46 () {56 vars in 1177 bytes} [Thu Jun 23 15:46:41 2022] GET /identity/v3/users/e9c7fd1e1ef54284a2b594ce3d58fc5a => generated 247 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:46:41.426594 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f94477b-a67d-445a-86c3-825c69265905 None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:46:41.427613 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f94477b-a67d-445a-86c3-825c69265905 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:46:41.427978 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f94477b-a67d-445a-86c3-825c69265905 None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:46:41.429963 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 14/30] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jun 23 15:46:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 15:46:41.430791 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f94477b-a67d-445a-86c3-825c69265905 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:46:41.434895 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-861d7cda-30a5-4f2d-b63f-76d4203769a8 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:46:41.436330 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-861d7cda-30a5-4f2d-b63f-76d4203769a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:46:41.519733 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 17/31] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jun 23 15:46:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 15:46:41.528156 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861d7cda-30a5-4f2d-b63f-76d4203769a8 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:46:41.562026 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f94477b-a67d-445a-86c3-825c69265905 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:46:41.607540 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a76d27-74eb-4d4e-af36-ded78bd34bf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:46:41.608231 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a76d27-74eb-4d4e-af36-ded78bd34bf5 None admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:46:41.608512 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a76d27-74eb-4d4e-af36-ded78bd34bf5 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:46:41.608786 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a76d27-74eb-4d4e-af36-ded78bd34bf5 None admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:46:41.610135 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a76d27-74eb-4d4e-af36-ded78bd34bf5 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:46:41.611902 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a76d27-74eb-4d4e-af36-ded78bd34bf5 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:46:41.622252 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:46:41.668552 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861d7cda-30a5-4f2d-b63f-76d4203769a8 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:46:41.708036 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1f5e22-aa81-427c-a369-6f433ea82600 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:46:41.709306 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1f5e22-aa81-427c-a369-6f433ea82600 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:46:41.709783 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1f5e22-aa81-427c-a369-6f433ea82600 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:46:41.710275 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1f5e22-aa81-427c-a369-6f433ea82600 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:46:41.711877 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1f5e22-aa81-427c-a369-6f433ea82600 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:46:41.713700 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1f5e22-aa81-427c-a369-6f433ea82600 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:46:41.715803 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 18/32] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Jun 23 15:46:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:46:41.717266 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cdf3ac-bbef-4d3c-bd55-2e633e3a2ee1 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:46:41.717741 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cdf3ac-bbef-4d3c-bd55-2e633e3a2ee1 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:46:41.718205 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cdf3ac-bbef-4d3c-bd55-2e633e3a2ee1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:46:41.743179 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-86cdf3ac-bbef-4d3c-bd55-2e633e3a2ee1 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:46:41.744970 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-86cdf3ac-bbef-4d3c-bd55-2e633e3a2ee1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:46:41.765890 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 15/33] 10.222.0.46 () {60 vars in 1131 bytes} [Thu Jun 23 15:46:41 2022] POST /identity/v3/roles => generated 219 bytes in 336 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 15:46:41.767797 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a76d27-74eb-4d4e-af36-ded78bd34bf5 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:46:41.809598 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:46:41.826356 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 19/34] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jun 23 15:46:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 15:46:41.827061 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cdf3ac-bbef-4d3c-bd55-2e633e3a2ee1 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:46:41.890534 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a76d27-74eb-4d4e-af36-ded78bd34bf5 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:46:41.926017 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:46:41.940010 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bd84ac-1e04-4d60-98aa-c5f246ab8083 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:46:41.941175 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bd84ac-1e04-4d60-98aa-c5f246ab8083 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:46:41.941680 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bd84ac-1e04-4d60-98aa-c5f246ab8083 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:46:41.942166 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bd84ac-1e04-4d60-98aa-c5f246ab8083 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:46:41.954589 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88bd84ac-1e04-4d60-98aa-c5f246ab8083 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:46:41.956597 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88bd84ac-1e04-4d60-98aa-c5f246ab8083 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:46:41.973576 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cdf3ac-bbef-4d3c-bd55-2e633e3a2ee1 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:46:41.976669 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-88bd84ac-1e04-4d60-98aa-c5f246ab8083 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 15:46:41.976669 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:46:41.976669 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:46:41.976669 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:46:41.976669 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:46:41.976669 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:46:41.976669 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:46:41.976669 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:46:41.976669 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:46:41.976669 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:46:41.976669 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:46:41.976669 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:46:41.976669 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 15:46:41.976669 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 15:46:41.976669 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 15:46:41.977539 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 15:46:41.977539 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:46:41.977539 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:46:41.977539 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:46:41.977539 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:46:41.977539 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:46:41.977539 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:46:41.977539 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:46:41.977539 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Jun 23 15:46:41.977539 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:46:41.977539 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:46:41.977539 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:46:41.977539 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:46:41.977539 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:46:41.977539 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:46:41.977539 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:46:41.977539 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:46:41.978326 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 15:46:41.978326 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 15:46:41.978326 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 15:46:41.978326 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Jun 23 15:46:41.980967 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 16/35] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Jun 23 15:46:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:46:41.982023 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bd84ac-1e04-4d60-98aa-c5f246ab8083 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:46:42.014978 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b33405a-e031-4f36-86fb-d1ae575188ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:46:42.015930 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b33405a-e031-4f36-86fb-d1ae575188ef None admin] REQUEST_METHOD: `PUT` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:46:42.016261 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b33405a-e031-4f36-86fb-d1ae575188ef None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:46:42.016478 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b33405a-e031-4f36-86fb-d1ae575188ef None admin] PATH_INFO: `/v3/domains/default/users/e9c7fd1e1ef54284a2b594ce3d58fc5a/roles/812d3c567c52469cb2b35709fbdc857c` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:46:42.018781 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b33405a-e031-4f36-86fb-d1ae575188ef None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=e9c7fd1e1ef54284a2b594ce3d58fc5a, role_id=812d3c567c52469cb2b35709fbdc857c)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:46:42.020288 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b33405a-e031-4f36-86fb-d1ae575188ef None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:46:42.087938 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:46:42.103537 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bd84ac-1e04-4d60-98aa-c5f246ab8083 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:46:42.140541 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec06c37-941d-4106-ba7e-87f9b2a56ceb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:46:42.141641 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec06c37-941d-4106-ba7e-87f9b2a56ceb None admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:46:42.142120 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec06c37-941d-4106-ba7e-87f9b2a56ceb None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:46:42.142629 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec06c37-941d-4106-ba7e-87f9b2a56ceb None admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:46:42.143619 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec06c37-941d-4106-ba7e-87f9b2a56ceb None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:46:42.145211 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec06c37-941d-4106-ba7e-87f9b2a56ceb None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:46:42.155323 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 20/36] 10.222.0.46 () {58 vars in 1361 bytes} [Thu Jun 23 15:46:41 2022] PUT /identity/v3/domains/default/users/e9c7fd1e1ef54284a2b594ce3d58fc5a/roles/812d3c567c52469cb2b35709fbdc857c => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 15:46:42.162818 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b33405a-e031-4f36-86fb-d1ae575188ef None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:46:42.171046 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 17/37] 10.222.0.46 () {60 vars in 1131 bytes} [Thu Jun 23 15:46:41 2022] POST /identity/v3/roles => generated 225 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 15:46:42.316605 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b33405a-e031-4f36-86fb-d1ae575188ef None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:46:42.352168 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:46:42.371838 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a09c1f-0cde-4771-b5b9-f6a223a2d9b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:46:42.373012 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a09c1f-0cde-4771-b5b9-f6a223a2d9b6 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:46:42.373499 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a09c1f-0cde-4771-b5b9-f6a223a2d9b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:46:42.373973 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a09c1f-0cde-4771-b5b9-f6a223a2d9b6 None admin] PATH_INFO: `/v3/domains` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:46:42.375019 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a09c1f-0cde-4771-b5b9-f6a223a2d9b6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:46:42.375562 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a09c1f-0cde-4771-b5b9-f6a223a2d9b6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:46:42.377701 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a09c1f-0cde-4771-b5b9-f6a223a2d9b6 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:46:42.403373 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 21/38] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Jun 23 15:46:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:46:42.409363 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec06c37-941d-4106-ba7e-87f9b2a56ceb None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:46:42.461540 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b99932e-1c07-4849-a7e4-0fb2578cccc7 None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:46:42.461856 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b99932e-1c07-4849-a7e4-0fb2578cccc7 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:46:42.462101 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b99932e-1c07-4849-a7e4-0fb2578cccc7 None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:46:42.464324 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 22/39] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jun 23 15:46:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 15:46:42.469763 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1385ac-f418-4f51-8098-14f6afa14653 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:46:42.470058 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1385ac-f418-4f51-8098-14f6afa14653 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:46:42.470344 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1385ac-f418-4f51-8098-14f6afa14653 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:46:42.489037 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-7b1385ac-f418-4f51-8098-14f6afa14653 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:46:42.490602 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-7b1385ac-f418-4f51-8098-14f6afa14653 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:46:42.534089 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec06c37-941d-4106-ba7e-87f9b2a56ceb None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:46:42.583899 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc1d42d-9778-4dd3-9b16-819e3a9806e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:46:42.584815 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc1d42d-9778-4dd3-9b16-819e3a9806e0 None admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:46:42.585221 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc1d42d-9778-4dd3-9b16-819e3a9806e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:46:42.585647 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc1d42d-9778-4dd3-9b16-819e3a9806e0 None admin] PATH_INFO: `/v3/projects` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:46:42.586534 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc1d42d-9778-4dd3-9b16-819e3a9806e0 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:46:42.588267 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc1d42d-9778-4dd3-9b16-819e3a9806e0 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:46:42.619153 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:46:42.630652 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 18/40] 10.222.0.46 () {60 vars in 1141 bytes} [Thu Jun 23 15:46:42 2022] POST /identity/v3/projects => generated 301 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 15:46:42.691170 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 23/41] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jun 23 15:46:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 15:46:42.715160 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc1d42d-9778-4dd3-9b16-819e3a9806e0 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:46:42.729667 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a43005d-a681-4f14-b93b-17cb5f89bc28 None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:46:42.729977 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a43005d-a681-4f14-b93b-17cb5f89bc28 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:46:42.730261 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a43005d-a681-4f14-b93b-17cb5f89bc28 None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:46:42.731963 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 24/42] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jun 23 15:46:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 15:46:42.737521 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367e8b06-76b9-4c34-ab07-9223f7ed6185 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:46:42.737841 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367e8b06-76b9-4c34-ab07-9223f7ed6185 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:46:42.738148 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367e8b06-76b9-4c34-ab07-9223f7ed6185 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:46:42.757253 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-367e8b06-76b9-4c34-ab07-9223f7ed6185 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:46:42.758496 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-367e8b06-76b9-4c34-ab07-9223f7ed6185 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:46:42.835959 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 25/43] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jun 23 15:46:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 15:46:42.850056 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc1d42d-9778-4dd3-9b16-819e3a9806e0 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:46:42.858970 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367e8b06-76b9-4c34-ab07-9223f7ed6185 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:46:42.893220 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970a8685-07ee-414c-b4a7-eac9640273fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:46:42.895271 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970a8685-07ee-414c-b4a7-eac9640273fd None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:46:42.895946 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970a8685-07ee-414c-b4a7-eac9640273fd None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:46:42.896706 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970a8685-07ee-414c-b4a7-eac9640273fd None admin] PATH_INFO: `/v3/domains/default` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:46:42.898678 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970a8685-07ee-414c-b4a7-eac9640273fd None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:46:42.900596 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970a8685-07ee-414c-b4a7-eac9640273fd None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:46:42.902831 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 19/44] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Jun 23 15:46:42 2022] GET /identity/v3/domains/default => generated 199 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:46:42.908708 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970a8685-07ee-414c-b4a7-eac9640273fd None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:46:42.909479 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:46:42.974830 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367e8b06-76b9-4c34-ab07-9223f7ed6185 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:46:42.981430 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:46:43.016609 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e22f54-9872-485d-8fa6-13ef710029df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:46:43.017289 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e22f54-9872-485d-8fa6-13ef710029df None admin] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:46:43.017574 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e22f54-9872-485d-8fa6-13ef710029df None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:46:43.017849 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e22f54-9872-485d-8fa6-13ef710029df None admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:46:43.018133 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970a8685-07ee-414c-b4a7-eac9640273fd None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:46:43.018603 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e22f54-9872-485d-8fa6-13ef710029df None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:46:43.020043 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e22f54-9872-485d-8fa6-13ef710029df None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:46:43.036513 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 26/45] 10.222.0.46 () {60 vars in 1131 bytes} [Thu Jun 23 15:46:42 2022] POST /identity/v3/roles => generated 223 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 15:46:43.058022 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56772a1-8f72-4062-8bc1-f74a3726ad74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:46:43.058763 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56772a1-8f72-4062-8bc1-f74a3726ad74 None admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:46:43.059061 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56772a1-8f72-4062-8bc1-f74a3726ad74 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:46:43.059345 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56772a1-8f72-4062-8bc1-f74a3726ad74 None admin] PATH_INFO: `/v3/projects` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:46:43.060101 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56772a1-8f72-4062-8bc1-f74a3726ad74 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:46:43.061654 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56772a1-8f72-4062-8bc1-f74a3726ad74 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:46:43.090840 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 20/46] 10.222.0.46 () {60 vars in 1141 bytes} [Thu Jun 23 15:46:42 2022] POST /identity/v3/projects => generated 312 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 15:46:44.382485 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbabe9d-280b-41cc-b25d-6fa7cf3de01f None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:46:44.383041 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbabe9d-280b-41cc-b25d-6fa7cf3de01f None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:46:44.383129 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbabe9d-280b-41cc-b25d-6fa7cf3de01f None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:46:44.384506 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 27/47] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jun 23 15:46:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 15:46:44.390456 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e30fcaf-fb5f-47bc-b493-efba9845aca7 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:46:44.390997 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e30fcaf-fb5f-47bc-b493-efba9845aca7 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:46:44.391081 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e30fcaf-fb5f-47bc-b493-efba9845aca7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:46:44.409284 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-0e30fcaf-fb5f-47bc-b493-efba9845aca7 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:46:44.410871 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-0e30fcaf-fb5f-47bc-b493-efba9845aca7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:46:44.482499 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 21/48] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jun 23 15:46:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 15:46:44.506949 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbabe9d-280b-41cc-b25d-6fa7cf3de01f None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:46:44.619249 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbabe9d-280b-41cc-b25d-6fa7cf3de01f None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:46:44.620828 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:46:44.655535 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fcbdde-06ad-4f32-a93a-95c168e51441 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:46:44.656250 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fcbdde-06ad-4f32-a93a-95c168e51441 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:46:44.656537 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fcbdde-06ad-4f32-a93a-95c168e51441 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:46:44.656817 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fcbdde-06ad-4f32-a93a-95c168e51441 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:46:44.668357 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fcbdde-06ad-4f32-a93a-95c168e51441 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 15:46:44.675975 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-97fcbdde-06ad-4f32-a93a-95c168e51441 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 15:46:44.675975 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:46:44.675975 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:46:44.675975 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:46:44.675975 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:46:44.675975 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:46:44.675975 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:46:44.675975 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:46:44.675975 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:46:44.675975 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:46:44.675975 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:46:44.675975 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:46:44.675975 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 15:46:44.675975 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 15:46:44.675975 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 15:46:44.675975 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 15:46:44.676838 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 15:46:44.676838 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 15:46:44.676838 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 15:46:44.676838 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 15:46:44.676838 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:46:44.676838 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:46:44.676838 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:46:44.676838 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:46:44.676838 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:46:44.676838 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:46:44.676838 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:46:44.676838 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Jun 23 15:46:44.676838 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:46:44.676838 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:46:44.676838 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:46:44.676838 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:46:44.677684 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:46:44.677684 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:46:44.677684 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:46:44.677684 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:46:44.677684 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 15:46:44.677684 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 15:46:44.677684 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 15:46:44.677684 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 15:46:44.677684 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 15:46:44.677684 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 15:46:44.677684 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 15:46:44.677684 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Jun 23 15:46:44.677684 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 28/49] 10.222.0.46 () {56 vars in 1096 bytes} [Thu Jun 23 15:46:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:46:44.683449 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e30fcaf-fb5f-47bc-b493-efba9845aca7 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:46:44.733252 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:46:44.785759 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e30fcaf-fb5f-47bc-b493-efba9845aca7 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:46:44.820923 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78f3ec7-4118-4ad3-84ec-8242bbaa5580 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:46:44.821612 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78f3ec7-4118-4ad3-84ec-8242bbaa5580 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:46:44.821899 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78f3ec7-4118-4ad3-84ec-8242bbaa5580 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:46:44.822204 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78f3ec7-4118-4ad3-84ec-8242bbaa5580 None admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:46:44.823004 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78f3ec7-4118-4ad3-84ec-8242bbaa5580 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:46:44.823315 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78f3ec7-4118-4ad3-84ec-8242bbaa5580 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:46:44.824724 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78f3ec7-4118-4ad3-84ec-8242bbaa5580 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:46:44.833510 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 22/50] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Jun 23 15:46:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:46:44.839637 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fcbdde-06ad-4f32-a93a-95c168e51441 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:46:44.943546 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fcbdde-06ad-4f32-a93a-95c168e51441 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:46:44.981452 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc44b9d-9f02-427f-a92d-27e08dfe5f77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:46:44.982345 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc44b9d-9f02-427f-a92d-27e08dfe5f77 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:46:44.982641 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc44b9d-9f02-427f-a92d-27e08dfe5f77 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:46:44.982952 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc44b9d-9f02-427f-a92d-27e08dfe5f77 None admin] PATH_INFO: `/v3/users/e9c7fd1e1ef54284a2b594ce3d58fc5a` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:46:44.984186 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc44b9d-9f02-427f-a92d-27e08dfe5f77 None admin] RBAC: Authorizing `identity:get_user(user_id=e9c7fd1e1ef54284a2b594ce3d58fc5a)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:46:44.985669 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc44b9d-9f02-427f-a92d-27e08dfe5f77 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:46:44.987553 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 29/51] 10.222.0.46 () {56 vars in 1177 bytes} [Thu Jun 23 15:46:44 2022] GET /identity/v3/users/e9c7fd1e1ef54284a2b594ce3d58fc5a => generated 247 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:46:44.992897 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78f3ec7-4118-4ad3-84ec-8242bbaa5580 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:46:45.152630 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d944253c-da13-46e8-ad57-b179a3456921 None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:46:45.152925 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d944253c-da13-46e8-ad57-b179a3456921 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:46:45.153217 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d944253c-da13-46e8-ad57-b179a3456921 None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:46:45.154669 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 30/52] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jun 23 15:46:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 15:46:45.160509 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d47e885-df35-4352-b9b4-3d9bb9a1d8f2 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:46:45.160800 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d47e885-df35-4352-b9b4-3d9bb9a1d8f2 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:46:45.161081 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d47e885-df35-4352-b9b4-3d9bb9a1d8f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:46:45.178261 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-1d47e885-df35-4352-b9b4-3d9bb9a1d8f2 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:46:45.179513 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-1d47e885-df35-4352-b9b4-3d9bb9a1d8f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:46:45.201586 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78f3ec7-4118-4ad3-84ec-8242bbaa5580 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:46:45.206290 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:46:45.216646 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:46:45.239544 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd1c240-ed98-4024-b28e-d95c15897f14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:46:45.240361 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd1c240-ed98-4024-b28e-d95c15897f14 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:46:45.240647 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd1c240-ed98-4024-b28e-d95c15897f14 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:46:45.240963 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd1c240-ed98-4024-b28e-d95c15897f14 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:46:45.242339 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd1c240-ed98-4024-b28e-d95c15897f14 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:46:45.243929 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd1c240-ed98-4024-b28e-d95c15897f14 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:46:45.246305 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 23/53] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Jun 23 15:46:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:46:45.252208 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd1c240-ed98-4024-b28e-d95c15897f14 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:46:45.258771 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 31/54] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jun 23 15:46:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 15:46:45.278755 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d47e885-df35-4352-b9b4-3d9bb9a1d8f2 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:46:45.358353 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd1c240-ed98-4024-b28e-d95c15897f14 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:46:45.381858 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d47e885-df35-4352-b9b4-3d9bb9a1d8f2 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:46:45.393811 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049adb5f-ad29-41bf-b41b-a0f59d0d6b6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:46:45.394487 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049adb5f-ad29-41bf-b41b-a0f59d0d6b6b None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:46:45.394809 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049adb5f-ad29-41bf-b41b-a0f59d0d6b6b None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:46:45.395091 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049adb5f-ad29-41bf-b41b-a0f59d0d6b6b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:46:45.395942 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049adb5f-ad29-41bf-b41b-a0f59d0d6b6b None admin] RBAC: Adding query filter params (role.id=812d3c567c52469cb2b35709fbdc857c, scope.domain.id=default, user.id=e9c7fd1e1ef54284a2b594ce3d58fc5a) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:46:45.396262 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049adb5f-ad29-41bf-b41b-a0f59d0d6b6b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:46:45.397727 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049adb5f-ad29-41bf-b41b-a0f59d0d6b6b None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:46:45.407807 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 24/55] 10.222.0.46 () {56 vars in 1322 bytes} [Thu Jun 23 15:46:45 2022] GET /identity/v3/role_assignments?user.id=e9c7fd1e1ef54284a2b594ce3d58fc5a&scope.domain.id=default&role.id=812d3c567c52469cb2b35709fbdc857c => generated 530 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:46:45.417135 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d455ea8-82f0-4f19-a39d-5d6bb38163ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:46:45.417974 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d455ea8-82f0-4f19-a39d-5d6bb38163ea None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:46:45.418260 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d455ea8-82f0-4f19-a39d-5d6bb38163ea None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:46:45.418539 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d455ea8-82f0-4f19-a39d-5d6bb38163ea None admin] PATH_INFO: `/v3/domains/default` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:46:45.419897 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d455ea8-82f0-4f19-a39d-5d6bb38163ea None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:46:45.421418 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d455ea8-82f0-4f19-a39d-5d6bb38163ea None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:46:45.423423 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 32/56] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Jun 23 15:46:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 15:46:45.429804 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049adb5f-ad29-41bf-b41b-a0f59d0d6b6b None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:46:45.481268 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:46:45.543111 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049adb5f-ad29-41bf-b41b-a0f59d0d6b6b None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:46:45.581993 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4144d53-f3c3-48ed-b41a-2eb78e1aa3cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:46:45.582679 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4144d53-f3c3-48ed-b41a-2eb78e1aa3cc None admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:46:45.583067 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4144d53-f3c3-48ed-b41a-2eb78e1aa3cc None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:46:45.583349 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4144d53-f3c3-48ed-b41a-2eb78e1aa3cc None admin] PATH_INFO: `/v3/projects` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:46:45.584118 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4144d53-f3c3-48ed-b41a-2eb78e1aa3cc None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:46:45.585590 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4144d53-f3c3-48ed-b41a-2eb78e1aa3cc None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:46:45.612562 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 25/57] 10.222.0.46 () {60 vars in 1140 bytes} [Thu Jun 23 15:46:45 2022] POST /identity/v3/projects => generated 298 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 15:46:47.661501 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea46084-9dd2-4b9a-bcb5-d72408d02511 None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:46:47.662008 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea46084-9dd2-4b9a-bcb5-d72408d02511 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:46:47.662117 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea46084-9dd2-4b9a-bcb5-d72408d02511 None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:46:47.663531 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 33/58] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jun 23 15:46:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 15:46:47.668811 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96becbcd-236d-490f-9b06-6c21bb3a6045 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:46:47.669245 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96becbcd-236d-490f-9b06-6c21bb3a6045 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:46:47.669389 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96becbcd-236d-490f-9b06-6c21bb3a6045 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:46:47.686471 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-96becbcd-236d-490f-9b06-6c21bb3a6045 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:46:47.687719 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-96becbcd-236d-490f-9b06-6c21bb3a6045 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:46:47.752147 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 26/59] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jun 23 15:46:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 15:46:47.777163 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea46084-9dd2-4b9a-bcb5-d72408d02511 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:46:47.804428 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:46:47.875845 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea46084-9dd2-4b9a-bcb5-d72408d02511 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:46:47.908963 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43588c0c-1333-4179-884c-eada037d0e3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:46:47.909764 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43588c0c-1333-4179-884c-eada037d0e3d None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:46:47.910053 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43588c0c-1333-4179-884c-eada037d0e3d None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:46:47.910332 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43588c0c-1333-4179-884c-eada037d0e3d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:46:47.911553 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43588c0c-1333-4179-884c-eada037d0e3d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:46:47.913000 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43588c0c-1333-4179-884c-eada037d0e3d None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:46:47.914618 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 34/60] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Jun 23 15:46:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:46:47.920749 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96becbcd-236d-490f-9b06-6c21bb3a6045 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:46:47.947419 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:46:48.026209 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96becbcd-236d-490f-9b06-6c21bb3a6045 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:46:48.058967 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fe3b3d-ff47-4ec0-8a66-c2e672de791d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:46:48.059654 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fe3b3d-ff47-4ec0-8a66-c2e672de791d None admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:46:48.059944 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fe3b3d-ff47-4ec0-8a66-c2e672de791d None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:46:48.060230 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fe3b3d-ff47-4ec0-8a66-c2e672de791d None admin] PATH_INFO: `/v3/users` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:46:48.060958 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fe3b3d-ff47-4ec0-8a66-c2e672de791d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:46:48.062424 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fe3b3d-ff47-4ec0-8a66-c2e672de791d None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:46:48.089015 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 27/61] 10.222.0.46 () {60 vars in 1132 bytes} [Thu Jun 23 15:46:47 2022] POST /identity/v3/users => generated 275 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 15:46:51.469607 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5886eec4-19f8-4594-ae94-242bd459263e None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:46:51.470501 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5886eec4-19f8-4594-ae94-242bd459263e None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:46:51.470963 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5886eec4-19f8-4594-ae94-242bd459263e None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:46:51.471378 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae6a6c2-90d0-40d8-9e8f-152b45653ee3 None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:46:51.472179 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae6a6c2-90d0-40d8-9e8f-152b45653ee3 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:46:51.472517 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 35/62] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jun 23 15:46:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 15:46:51.472875 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae6a6c2-90d0-40d8-9e8f-152b45653ee3 None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:46:51.474434 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 28/63] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jun 23 15:46:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 15:46:51.477817 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ac866a-32dd-47d7-b2d5-6a724376a4a2 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:46:51.478177 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ac866a-32dd-47d7-b2d5-6a724376a4a2 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:46:51.479012 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ac866a-32dd-47d7-b2d5-6a724376a4a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:46:51.481285 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a00985-1546-4396-9b30-df6bdaa6b67f None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:46:51.481855 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a00985-1546-4396-9b30-df6bdaa6b67f None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:46:51.482324 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a00985-1546-4396-9b30-df6bdaa6b67f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:46:51.499359 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-e9a00985-1546-4396-9b30-df6bdaa6b67f None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:46:51.500716 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-09ac866a-32dd-47d7-b2d5-6a724376a4a2 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:46:51.501250 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-e9a00985-1546-4396-9b30-df6bdaa6b67f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:46:51.501900 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-09ac866a-32dd-47d7-b2d5-6a724376a4a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:46:51.601853 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 29/64] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jun 23 15:46:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 15:46:51.609918 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772f5f56-ceb5-423a-92db-409fc08b3474 None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:46:51.609918 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772f5f56-ceb5-423a-92db-409fc08b3474 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:46:51.609918 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772f5f56-ceb5-423a-92db-409fc08b3474 None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:46:51.609918 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 30/65] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jun 23 15:46:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 15:46:51.611524 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 36/66] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jun 23 15:46:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 15:46:51.611653 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd701fa2-33ba-401c-a669-f8da503f0c70 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:46:51.612108 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd701fa2-33ba-401c-a669-f8da503f0c70 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:46:51.612608 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd701fa2-33ba-401c-a669-f8da503f0c70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:46:51.623690 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c60ba7-5adb-4926-ade8-ed8dde77781e None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:46:51.624112 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c60ba7-5adb-4926-ade8-ed8dde77781e None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:46:51.624487 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c60ba7-5adb-4926-ade8-ed8dde77781e None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:46:51.625980 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 37/67] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jun 23 15:46:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 15:46:51.631252 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73379aec-4d56-442f-ab02-c18164bb253a None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:46:51.631663 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73379aec-4d56-442f-ab02-c18164bb253a None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:46:51.632039 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73379aec-4d56-442f-ab02-c18164bb253a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:46:51.641071 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-dd701fa2-33ba-401c-a669-f8da503f0c70 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:46:51.642295 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-dd701fa2-33ba-401c-a669-f8da503f0c70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:46:51.652251 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-73379aec-4d56-442f-ab02-c18164bb253a None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:46:51.652416 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:46:51.653400 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-73379aec-4d56-442f-ab02-c18164bb253a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:46:51.675073 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:46:51.722060 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 31/68] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jun 23 15:46:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 15:46:51.724308 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd701fa2-33ba-401c-a669-f8da503f0c70 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:46:51.728667 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 38/69] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jun 23 15:46:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 15:46:51.729756 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73379aec-4d56-442f-ab02-c18164bb253a None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:46:51.836971 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73379aec-4d56-442f-ab02-c18164bb253a None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:46:51.844513 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd701fa2-33ba-401c-a669-f8da503f0c70 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:46:51.872661 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46b2c53-297c-4b1f-9846-dffe78d9d009 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:46:51.873472 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46b2c53-297c-4b1f-9846-dffe78d9d009 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:46:51.873757 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46b2c53-297c-4b1f-9846-dffe78d9d009 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:46:51.874035 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46b2c53-297c-4b1f-9846-dffe78d9d009 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:46:51.881547 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab5ad5d-1c2e-43d2-93cc-08ce6943f35d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:46:51.882362 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab5ad5d-1c2e-43d2-93cc-08ce6943f35d None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:46:51.882652 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab5ad5d-1c2e-43d2-93cc-08ce6943f35d None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:46:51.882970 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab5ad5d-1c2e-43d2-93cc-08ce6943f35d None admin] PATH_INFO: `/v3/roles/member` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:46:51.885248 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46b2c53-297c-4b1f-9846-dffe78d9d009 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 15:46:51.892992 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-e46b2c53-297c-4b1f-9846-dffe78d9d009 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 15:46:51.892992 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:46:51.892992 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:46:51.892992 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:46:51.892992 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:46:51.892992 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:46:51.892992 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:46:51.892992 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:46:51.892992 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:46:51.892992 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:46:51.892992 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:46:51.892992 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:46:51.892992 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 15:46:51.892992 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 15:46:51.892992 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 15:46:51.892992 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 15:46:51.893844 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 15:46:51.893844 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 15:46:51.893844 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 15:46:51.893844 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 15:46:51.893844 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:46:51.893844 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:46:51.893844 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:46:51.893844 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:46:51.893844 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:46:51.893844 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:46:51.893844 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:46:51.893844 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Jun 23 15:46:51.893844 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:46:51.893844 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:46:51.893844 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:46:51.893844 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:46:51.899578 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:46:51.899578 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:46:51.899578 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:46:51.899578 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:46:51.899578 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 15:46:51.899578 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 15:46:51.899578 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 15:46:51.899578 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 15:46:51.899578 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 15:46:51.899578 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 15:46:51.899578 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 15:46:51.899578 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Jun 23 15:46:51.899578 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 39/70] 10.222.0.46 () {56 vars in 1096 bytes} [Thu Jun 23 15:46:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:46:51.899578 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46b2c53-297c-4b1f-9846-dffe78d9d009 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:46:51.902726 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab5ad5d-1c2e-43d2-93cc-08ce6943f35d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 15:46:51.911476 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-bab5ad5d-1c2e-43d2-93cc-08ce6943f35d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 23 15:46:51.911476 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:46:51.911476 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:46:51.911476 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:46:51.911476 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:46:51.911476 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:46:51.911476 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:46:51.911476 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:46:51.911476 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:46:51.911476 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:46:51.911476 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:46:51.911476 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:46:51.911476 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 15:46:51.911476 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 15:46:51.911476 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 15:46:51.911476 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 15:46:51.912406 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 15:46:51.912406 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 15:46:51.912406 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 15:46:51.912406 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 15:46:51.912406 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:46:51.912406 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:46:51.912406 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:46:51.912406 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:46:51.912406 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:46:51.912406 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:46:51.912406 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:46:51.912406 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Jun 23 15:46:51.912406 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:46:51.912406 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:46:51.912406 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:46:51.912406 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:46:51.913771 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:46:51.913771 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:46:51.913771 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:46:51.913771 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:46:51.913771 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 15:46:51.913771 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 15:46:51.913771 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 15:46:51.913771 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 15:46:51.913771 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 15:46:51.913771 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 15:46:51.913771 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 23 15:46:51.913771 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Jun 23 15:46:51.913771 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 32/71] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Jun 23 15:46:51 2022] GET /identity/v3/roles/member => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:46:51.913771 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab5ad5d-1c2e-43d2-93cc-08ce6943f35d None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:46:51.917242 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:46:51.920943 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:46:52.008565 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46b2c53-297c-4b1f-9846-dffe78d9d009 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:46:52.023614 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab5ad5d-1c2e-43d2-93cc-08ce6943f35d None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:46:52.046346 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461d8748-4af2-4c16-8881-4786508c124b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:46:52.047391 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461d8748-4af2-4c16-8881-4786508c124b None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:46:52.047685 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461d8748-4af2-4c16-8881-4786508c124b None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:46:52.047969 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461d8748-4af2-4c16-8881-4786508c124b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:46:52.060518 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461d8748-4af2-4c16-8881-4786508c124b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 15:46:52.065117 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe864ef3-7fef-4d28-b548-4e769f463096 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:46:52.065932 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe864ef3-7fef-4d28-b548-4e769f463096 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:46:52.066218 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe864ef3-7fef-4d28-b548-4e769f463096 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:46:52.066495 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe864ef3-7fef-4d28-b548-4e769f463096 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:46:52.069026 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-461d8748-4af2-4c16-8881-4786508c124b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 15:46:52.069026 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:46:52.069026 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:46:52.069026 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:46:52.069026 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:46:52.069026 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:46:52.069026 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:46:52.069026 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:46:52.069026 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:46:52.069026 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:46:52.069026 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:46:52.069026 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:46:52.069026 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 15:46:52.069026 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 15:46:52.069026 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 15:46:52.069026 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 15:46:52.069883 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 15:46:52.069883 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 15:46:52.069883 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 15:46:52.069883 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 15:46:52.069883 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:46:52.069883 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:46:52.069883 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:46:52.069883 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:46:52.069883 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:46:52.069883 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:46:52.069883 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:46:52.069883 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Jun 23 15:46:52.069883 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:46:52.069883 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:46:52.069883 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:46:52.069883 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:46:52.070632 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:46:52.070632 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:46:52.070632 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:46:52.070632 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:46:52.070632 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 15:46:52.070632 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 15:46:52.070632 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 15:46:52.070632 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 15:46:52.070632 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 15:46:52.070632 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 15:46:52.070632 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 15:46:52.070632 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Jun 23 15:46:52.070632 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 40/72] 10.222.0.46 () {56 vars in 1114 bytes} [Thu Jun 23 15:46:51 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:46:52.071279 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461d8748-4af2-4c16-8881-4786508c124b None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:46:52.079972 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe864ef3-7fef-4d28-b548-4e769f463096 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 15:46:52.087837 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-fe864ef3-7fef-4d28-b548-4e769f463096 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 23 15:46:52.087837 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:46:52.087837 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:46:52.087837 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:46:52.087837 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:46:52.087837 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:46:52.087837 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:46:52.087837 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:46:52.087837 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:46:52.087837 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:46:52.087837 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:46:52.087837 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:46:52.087837 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 15:46:52.087837 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 15:46:52.087837 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 15:46:52.087837 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 15:46:52.088852 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 15:46:52.088852 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 15:46:52.088852 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 15:46:52.088852 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 15:46:52.088852 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:46:52.088852 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:46:52.088852 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:46:52.088852 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:46:52.088852 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:46:52.088852 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:46:52.088852 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:46:52.088852 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Jun 23 15:46:52.088852 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:46:52.088852 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:46:52.088852 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:46:52.088852 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:46:52.089691 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:46:52.089691 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:46:52.089691 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:46:52.089691 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:46:52.089691 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 15:46:52.089691 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 15:46:52.089691 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 15:46:52.089691 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 15:46:52.089691 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 15:46:52.089691 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 15:46:52.089691 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 23 15:46:52.089691 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Jun 23 15:46:52.089691 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 33/73] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Jun 23 15:46:51 2022] GET /identity/v3/roles/member => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:46:52.090320 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e09a27-428b-428f-8e13-40434e47c34b None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:46:52.090589 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e09a27-428b-428f-8e13-40434e47c34b None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:46:52.090895 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e09a27-428b-428f-8e13-40434e47c34b None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:46:52.092168 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 34/74] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jun 23 15:46:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 15:46:52.092959 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e09a27-428b-428f-8e13-40434e47c34b None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:46:52.152317 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:46:52.157663 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:46:52.176227 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461d8748-4af2-4c16-8881-4786508c124b None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:46:52.198009 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e09a27-428b-428f-8e13-40434e47c34b None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:46:52.211079 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cf3fa0-feef-446b-b9e0-74a70275c502 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:46:52.211755 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cf3fa0-feef-446b-b9e0-74a70275c502 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:46:52.212043 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cf3fa0-feef-446b-b9e0-74a70275c502 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:46:52.212323 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cf3fa0-feef-446b-b9e0-74a70275c502 None admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:46:52.213113 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77cf3fa0-feef-446b-b9e0-74a70275c502 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:46:52.213426 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77cf3fa0-feef-446b-b9e0-74a70275c502 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:46:52.214865 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77cf3fa0-feef-446b-b9e0-74a70275c502 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:46:52.223156 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 41/75] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Jun 23 15:46:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:46:52.223967 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cf3fa0-feef-446b-b9e0-74a70275c502 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:46:52.233069 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcd29dc-96fe-48f1-88fd-bc82236da50b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:46:52.233752 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcd29dc-96fe-48f1-88fd-bc82236da50b None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:46:52.234036 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcd29dc-96fe-48f1-88fd-bc82236da50b None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:46:52.234312 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcd29dc-96fe-48f1-88fd-bc82236da50b None admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:46:52.235127 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbcd29dc-96fe-48f1-88fd-bc82236da50b None admin] RBAC: Adding query filter params (name=member) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:46:52.235436 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbcd29dc-96fe-48f1-88fd-bc82236da50b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:46:52.236825 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbcd29dc-96fe-48f1-88fd-bc82236da50b None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:46:52.246612 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 35/76] 10.222.0.46 () {56 vars in 1101 bytes} [Thu Jun 23 15:46:52 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:46:52.247484 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcd29dc-96fe-48f1-88fd-bc82236da50b None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:46:52.331936 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cf3fa0-feef-446b-b9e0-74a70275c502 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:46:52.349601 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcd29dc-96fe-48f1-88fd-bc82236da50b None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:46:52.365848 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be145bb-2d89-4680-974f-f3b70fe7712a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:46:52.366548 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be145bb-2d89-4680-974f-f3b70fe7712a None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:46:52.366865 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be145bb-2d89-4680-974f-f3b70fe7712a None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:46:52.367152 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be145bb-2d89-4680-974f-f3b70fe7712a None admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:46:52.367911 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be145bb-2d89-4680-974f-f3b70fe7712a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:46:52.368216 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be145bb-2d89-4680-974f-f3b70fe7712a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:46:52.370114 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be145bb-2d89-4680-974f-f3b70fe7712a None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:46:52.373876 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:46:52.375704 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:46:52.380159 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 42/77] 10.222.0.46 () {56 vars in 1111 bytes} [Thu Jun 23 15:46:52 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:46:52.381803 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20fa2ba-a588-4ba8-bc52-3c68b688cf57 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:46:52.382753 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20fa2ba-a588-4ba8-bc52-3c68b688cf57 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:46:52.383089 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20fa2ba-a588-4ba8-bc52-3c68b688cf57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:46:52.386624 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9f54c1-f9f7-4db8-96b9-2f22fc932f10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:46:52.387339 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9f54c1-f9f7-4db8-96b9-2f22fc932f10 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:46:52.387633 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9f54c1-f9f7-4db8-96b9-2f22fc932f10 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:46:52.387944 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9f54c1-f9f7-4db8-96b9-2f22fc932f10 None admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:46:52.388711 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9f54c1-f9f7-4db8-96b9-2f22fc932f10 None admin] RBAC: Adding query filter params (name=member) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:46:52.389025 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9f54c1-f9f7-4db8-96b9-2f22fc932f10 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:46:52.390411 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9f54c1-f9f7-4db8-96b9-2f22fc932f10 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:46:52.398723 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 36/78] 10.222.0.46 () {56 vars in 1101 bytes} [Thu Jun 23 15:46:52 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:46:52.399541 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9f54c1-f9f7-4db8-96b9-2f22fc932f10 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:46:52.404457 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-f20fa2ba-a588-4ba8-bc52-3c68b688cf57 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:46:52.405763 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-f20fa2ba-a588-4ba8-bc52-3c68b688cf57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:46:52.474723 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 43/79] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jun 23 15:46:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 15:46:52.475609 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20fa2ba-a588-4ba8-bc52-3c68b688cf57 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:46:52.503709 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9f54c1-f9f7-4db8-96b9-2f22fc932f10 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:46:52.537422 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c161a8-59a7-4611-a720-ac0a1c6f475a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:46:52.538315 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c161a8-59a7-4611-a720-ac0a1c6f475a None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:46:52.538598 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c161a8-59a7-4611-a720-ac0a1c6f475a None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:46:52.538906 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c161a8-59a7-4611-a720-ac0a1c6f475a None admin] PATH_INFO: `/v3/users/e9c7fd1e1ef54284a2b594ce3d58fc5a` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:46:52.540121 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c161a8-59a7-4611-a720-ac0a1c6f475a None admin] RBAC: Authorizing `identity:get_user(user_id=e9c7fd1e1ef54284a2b594ce3d58fc5a)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:46:52.542137 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c161a8-59a7-4611-a720-ac0a1c6f475a None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:46:52.543806 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 37/80] 10.222.0.46 () {56 vars in 1177 bytes} [Thu Jun 23 15:46:52 2022] GET /identity/v3/users/e9c7fd1e1ef54284a2b594ce3d58fc5a => generated 247 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:46:52.544571 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c161a8-59a7-4611-a720-ac0a1c6f475a None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:46:52.586363 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20fa2ba-a588-4ba8-bc52-3c68b688cf57 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:46:52.604562 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:46:52.616038 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:46:52.623185 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60809fa-0b84-423b-870f-2c85ba207a50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:46:52.624076 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60809fa-0b84-423b-870f-2c85ba207a50 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:46:52.624368 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60809fa-0b84-423b-870f-2c85ba207a50 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:46:52.624650 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60809fa-0b84-423b-870f-2c85ba207a50 None admin] PATH_INFO: `/v3/users/160079811a0e42009ddb10073ea2cd06` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:46:52.639095 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60809fa-0b84-423b-870f-2c85ba207a50 None admin] RBAC: Authorizing `identity:get_user(user_id=160079811a0e42009ddb10073ea2cd06)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:46:52.640581 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60809fa-0b84-423b-870f-2c85ba207a50 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:46:52.642304 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 44/81] 10.222.0.46 () {56 vars in 1177 bytes} [Thu Jun 23 15:46:52 2022] GET /identity/v3/users/160079811a0e42009ddb10073ea2cd06 => generated 275 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:46:52.643125 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60809fa-0b84-423b-870f-2c85ba207a50 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:46:52.652378 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c161a8-59a7-4611-a720-ac0a1c6f475a None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:46:52.687287 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0703a0-4a41-4515-a1bd-ce4b142ecbdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:46:52.688025 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0703a0-4a41-4515-a1bd-ce4b142ecbdc None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:46:52.688317 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0703a0-4a41-4515-a1bd-ce4b142ecbdc None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:46:52.688598 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0703a0-4a41-4515-a1bd-ce4b142ecbdc None admin] PATH_INFO: `/v3/users/160079811a0e42009ddb10073ea2cd06` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:46:52.689830 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0703a0-4a41-4515-a1bd-ce4b142ecbdc None admin] RBAC: Authorizing `identity:get_user(user_id=160079811a0e42009ddb10073ea2cd06)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:46:52.691359 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0703a0-4a41-4515-a1bd-ce4b142ecbdc None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:46:52.693010 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 38/82] 10.222.0.46 () {56 vars in 1177 bytes} [Thu Jun 23 15:46:52 2022] GET /identity/v3/users/160079811a0e42009ddb10073ea2cd06 => generated 275 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:46:52.693783 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0703a0-4a41-4515-a1bd-ce4b142ecbdc None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:46:52.746078 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60809fa-0b84-423b-870f-2c85ba207a50 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:46:52.782386 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cca04d-6a3e-4440-abf3-c4a133c7b528 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:46:52.783296 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cca04d-6a3e-4440-abf3-c4a133c7b528 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:46:52.783576 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cca04d-6a3e-4440-abf3-c4a133c7b528 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:46:52.783875 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cca04d-6a3e-4440-abf3-c4a133c7b528 None admin] PATH_INFO: `/v3/users/160079811a0e42009ddb10073ea2cd06` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:46:52.785073 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cca04d-6a3e-4440-abf3-c4a133c7b528 None admin] RBAC: Authorizing `identity:get_user(user_id=160079811a0e42009ddb10073ea2cd06)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:46:52.786543 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cca04d-6a3e-4440-abf3-c4a133c7b528 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:46:52.788215 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 45/83] 10.222.0.46 () {56 vars in 1177 bytes} [Thu Jun 23 15:46:52 2022] GET /identity/v3/users/160079811a0e42009ddb10073ea2cd06 => generated 275 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:46:52.788976 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cca04d-6a3e-4440-abf3-c4a133c7b528 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:46:52.795739 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0703a0-4a41-4515-a1bd-ce4b142ecbdc None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:46:52.817974 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:46:52.830605 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5d3ea4-3ffd-44e7-88d8-55538cea7dcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:46:52.831444 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5d3ea4-3ffd-44e7-88d8-55538cea7dcc None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:46:52.831739 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5d3ea4-3ffd-44e7-88d8-55538cea7dcc None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:46:52.832025 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5d3ea4-3ffd-44e7-88d8-55538cea7dcc None admin] PATH_INFO: `/v3/domains/default` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:46:52.833230 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5d3ea4-3ffd-44e7-88d8-55538cea7dcc None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:46:52.834719 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5d3ea4-3ffd-44e7-88d8-55538cea7dcc None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:46:52.836587 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 39/84] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Jun 23 15:46:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:46:52.837154 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5d3ea4-3ffd-44e7-88d8-55538cea7dcc None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:46:52.850481 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:46:52.895144 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cca04d-6a3e-4440-abf3-c4a133c7b528 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:46:52.930315 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b9d060-6b45-4c19-a84a-26fc3af0f792 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:46:52.931021 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b9d060-6b45-4c19-a84a-26fc3af0f792 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:46:52.931341 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b9d060-6b45-4c19-a84a-26fc3af0f792 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:46:52.931626 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b9d060-6b45-4c19-a84a-26fc3af0f792 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:46:52.939206 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5d3ea4-3ffd-44e7-88d8-55538cea7dcc None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:46:52.973779 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7c0fd4-ac77-4df7-9b41-b9f667b11254 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:46:52.974466 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7c0fd4-ac77-4df7-9b41-b9f667b11254 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:46:52.974781 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7c0fd4-ac77-4df7-9b41-b9f667b11254 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:46:52.975067 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7c0fd4-ac77-4df7-9b41-b9f667b11254 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:46:53.001611 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b9d060-6b45-4c19-a84a-26fc3af0f792 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:46:53.003145 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b9d060-6b45-4c19-a84a-26fc3af0f792 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:46:53.034577 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:46:53.045765 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7c0fd4-ac77-4df7-9b41-b9f667b11254 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:46:53.047297 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7c0fd4-ac77-4df7-9b41-b9f667b11254 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:46:53.060163 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:46:53.105202 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 46/85] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Jun 23 15:46:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 15:46:53.105984 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b9d060-6b45-4c19-a84a-26fc3af0f792 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:46:53.153473 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 40/86] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Jun 23 15:46:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 15:46:53.154357 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7c0fd4-ac77-4df7-9b41-b9f667b11254 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:46:53.207736 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b9d060-6b45-4c19-a84a-26fc3af0f792 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:46:53.242770 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c34ce8-86e4-43fc-8d73-92768b04e1a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:46:53.243802 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c34ce8-86e4-43fc-8d73-92768b04e1a3 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:46:53.244467 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c34ce8-86e4-43fc-8d73-92768b04e1a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:46:53.244850 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c34ce8-86e4-43fc-8d73-92768b04e1a3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:46:53.245384 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:46:53.255822 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7c0fd4-ac77-4df7-9b41-b9f667b11254 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:46:53.272478 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:46:53.294183 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3cf30d-d932-4aa2-aaf5-19e9713f886e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:46:53.295012 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3cf30d-d932-4aa2-aaf5-19e9713f886e None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:46:53.295249 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3cf30d-d932-4aa2-aaf5-19e9713f886e None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:46:53.295536 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3cf30d-d932-4aa2-aaf5-19e9713f886e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:46:53.325775 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c34ce8-86e4-43fc-8d73-92768b04e1a3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:46:53.327314 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c34ce8-86e4-43fc-8d73-92768b04e1a3 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:46:53.364065 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3cf30d-d932-4aa2-aaf5-19e9713f886e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:46:53.365467 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3cf30d-d932-4aa2-aaf5-19e9713f886e None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:46:53.428804 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 47/87] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Jun 23 15:46:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 15:46:53.429885 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c34ce8-86e4-43fc-8d73-92768b04e1a3 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:46:53.456731 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:46:53.467591 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 41/88] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Jun 23 15:46:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 15:46:53.468487 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3cf30d-d932-4aa2-aaf5-19e9713f886e None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:46:53.491079 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:46:53.545371 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c34ce8-86e4-43fc-8d73-92768b04e1a3 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:46:53.582182 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3cf30d-d932-4aa2-aaf5-19e9713f886e None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:46:53.585835 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c3de97-843c-4953-b1f4-fac7b843fdc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:46:53.586511 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c3de97-843c-4953-b1f4-fac7b843fdc9 None admin] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:46:53.586826 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c3de97-843c-4953-b1f4-fac7b843fdc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:46:53.587106 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c3de97-843c-4953-b1f4-fac7b843fdc9 None admin] PATH_INFO: `/v3/users` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:46:53.587858 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c3de97-843c-4953-b1f4-fac7b843fdc9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:46:53.589323 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c3de97-843c-4953-b1f4-fac7b843fdc9 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:46:53.616735 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 48/89] 10.222.0.46 () {60 vars in 1132 bytes} [Thu Jun 23 15:46:53 2022] POST /identity/v3/users => generated 289 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 15:46:53.617510 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c3de97-843c-4953-b1f4-fac7b843fdc9 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:46:53.631382 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b599f3a-026f-4b6f-ae08-b0dddbc45c40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:46:53.632331 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b599f3a-026f-4b6f-ae08-b0dddbc45c40 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:46:53.632617 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b599f3a-026f-4b6f-ae08-b0dddbc45c40 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:46:53.632917 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b599f3a-026f-4b6f-ae08-b0dddbc45c40 None admin] PATH_INFO: `/v3/projects/27f259ec3bd04186a70598926b09aa56` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:46:53.635092 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b599f3a-026f-4b6f-ae08-b0dddbc45c40 None admin] RBAC: Authorizing `identity:get_project(project_id=27f259ec3bd04186a70598926b09aa56)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:46:53.637270 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b599f3a-026f-4b6f-ae08-b0dddbc45c40 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:46:53.644253 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 42/90] 10.222.0.46 () {56 vars in 1186 bytes} [Thu Jun 23 15:46:53 2022] GET /identity/v3/projects/27f259ec3bd04186a70598926b09aa56 => generated 298 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:46:53.644253 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b599f3a-026f-4b6f-ae08-b0dddbc45c40 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:46:53.719263 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:46:53.756423 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c3de97-843c-4953-b1f4-fac7b843fdc9 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:46:53.772611 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b599f3a-026f-4b6f-ae08-b0dddbc45c40 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:46:53.774253 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:46:53.803158 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f8b68e-f4e5-4b89-9014-ca528ea7206c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:46:53.804854 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f8b68e-f4e5-4b89-9014-ca528ea7206c None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:46:53.804854 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f8b68e-f4e5-4b89-9014-ca528ea7206c None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:46:53.805803 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f8b68e-f4e5-4b89-9014-ca528ea7206c None admin] PATH_INFO: `/v3/projects/27f259ec3bd04186a70598926b09aa56` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:46:53.808013 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f8b68e-f4e5-4b89-9014-ca528ea7206c None admin] RBAC: Authorizing `identity:get_project(project_id=27f259ec3bd04186a70598926b09aa56)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:46:53.810223 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f8b68e-f4e5-4b89-9014-ca528ea7206c None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:46:53.812151 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 49/91] 10.222.0.46 () {56 vars in 1186 bytes} [Thu Jun 23 15:46:53 2022] GET /identity/v3/projects/27f259ec3bd04186a70598926b09aa56 => generated 298 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:46:53.812264 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f8b68e-f4e5-4b89-9014-ca528ea7206c None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:46:53.838454 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6ee0d0-35eb-489a-b02e-ad48d39188ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:46:53.839514 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6ee0d0-35eb-489a-b02e-ad48d39188ea None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:46:53.839797 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6ee0d0-35eb-489a-b02e-ad48d39188ea None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:46:53.840073 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6ee0d0-35eb-489a-b02e-ad48d39188ea None admin] PATH_INFO: `/v3/projects/27f259ec3bd04186a70598926b09aa56` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:46:53.841290 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6ee0d0-35eb-489a-b02e-ad48d39188ea None admin] RBAC: Authorizing `identity:get_project(project_id=27f259ec3bd04186a70598926b09aa56)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:46:53.842793 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6ee0d0-35eb-489a-b02e-ad48d39188ea None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:46:53.844853 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 43/92] 10.222.0.46 () {56 vars in 1186 bytes} [Thu Jun 23 15:46:53 2022] GET /identity/v3/projects/27f259ec3bd04186a70598926b09aa56 => generated 298 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:46:53.845272 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6ee0d0-35eb-489a-b02e-ad48d39188ea None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:46:53.942960 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f8b68e-f4e5-4b89-9014-ca528ea7206c None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:46:53.984221 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6ee0d0-35eb-489a-b02e-ad48d39188ea None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:46:53.984405 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2ff3ba-b0c0-46b0-8fd2-dc90a2c6f142 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:46:53.985155 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2ff3ba-b0c0-46b0-8fd2-dc90a2c6f142 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:46:53.985441 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2ff3ba-b0c0-46b0-8fd2-dc90a2c6f142 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:46:53.985828 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2ff3ba-b0c0-46b0-8fd2-dc90a2c6f142 None admin] PATH_INFO: `/v3/projects/34606c7f36fa455d8daf5ec64dafa1d4` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:46:53.988258 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2ff3ba-b0c0-46b0-8fd2-dc90a2c6f142 None admin] RBAC: Authorizing `identity:get_project(project_id=34606c7f36fa455d8daf5ec64dafa1d4)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:46:53.989758 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2ff3ba-b0c0-46b0-8fd2-dc90a2c6f142 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:46:53.991742 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 50/93] 10.222.0.46 () {56 vars in 1186 bytes} [Thu Jun 23 15:46:53 2022] GET /identity/v3/projects/34606c7f36fa455d8daf5ec64dafa1d4 => generated 312 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:46:53.992339 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2ff3ba-b0c0-46b0-8fd2-dc90a2c6f142 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:46:54.029677 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:46:54.043037 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56649e05-8aeb-4597-b96d-b6c6e068ddf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:46:54.043975 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56649e05-8aeb-4597-b96d-b6c6e068ddf7 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:46:54.044273 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56649e05-8aeb-4597-b96d-b6c6e068ddf7 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:46:54.045017 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56649e05-8aeb-4597-b96d-b6c6e068ddf7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:46:54.046171 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56649e05-8aeb-4597-b96d-b6c6e068ddf7 None admin] RBAC: Adding query filter params (role.id=812d3c567c52469cb2b35709fbdc857c, scope.project.id=27f259ec3bd04186a70598926b09aa56, user.id=e9c7fd1e1ef54284a2b594ce3d58fc5a) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:46:54.046500 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56649e05-8aeb-4597-b96d-b6c6e068ddf7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:46:54.047519 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:46:54.049006 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56649e05-8aeb-4597-b96d-b6c6e068ddf7 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:46:54.059113 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 44/94] 10.222.0.46 () {56 vars in 1374 bytes} [Thu Jun 23 15:46:53 2022] GET /identity/v3/role_assignments?user.id=e9c7fd1e1ef54284a2b594ce3d58fc5a&scope.project.id=27f259ec3bd04186a70598926b09aa56&role.id=812d3c567c52469cb2b35709fbdc857c => generated 259 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:46:54.059521 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56649e05-8aeb-4597-b96d-b6c6e068ddf7 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:46:54.167180 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2ff3ba-b0c0-46b0-8fd2-dc90a2c6f142 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:46:54.190736 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56649e05-8aeb-4597-b96d-b6c6e068ddf7 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:46:54.238118 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e079182e-c87f-4191-a2a7-d72315660cee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:46:54.238780 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e079182e-c87f-4191-a2a7-d72315660cee None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:46:54.240325 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e079182e-c87f-4191-a2a7-d72315660cee None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:46:54.242253 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e079182e-c87f-4191-a2a7-d72315660cee None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:46:54.244353 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e079182e-c87f-4191-a2a7-d72315660cee None admin] RBAC: Adding query filter params (role.id=d4d557a3ec4e434f8e83499c5fcc7136, scope.project.id=27f259ec3bd04186a70598926b09aa56, user.id=160079811a0e42009ddb10073ea2cd06) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:46:54.244683 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e079182e-c87f-4191-a2a7-d72315660cee None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:46:54.246263 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e079182e-c87f-4191-a2a7-d72315660cee None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:46:54.256213 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 45/95] 10.222.0.46 () {56 vars in 1374 bytes} [Thu Jun 23 15:46:54 2022] GET /identity/v3/role_assignments?user.id=160079811a0e42009ddb10073ea2cd06&scope.project.id=27f259ec3bd04186a70598926b09aa56&role.id=d4d557a3ec4e434f8e83499c5fcc7136 => generated 259 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:46:54.257136 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e079182e-c87f-4191-a2a7-d72315660cee None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:46:54.283498 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ff3ce7-0e86-44bb-9661-f6d5e9babaed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:46:54.284188 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ff3ce7-0e86-44bb-9661-f6d5e9babaed None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:46:54.284472 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ff3ce7-0e86-44bb-9661-f6d5e9babaed None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:46:54.284748 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ff3ce7-0e86-44bb-9661-f6d5e9babaed None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:46:54.285609 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ff3ce7-0e86-44bb-9661-f6d5e9babaed None admin] RBAC: Adding query filter params (role.id=e2a5bbf3200a4d4091d72bc7ff78c34d, scope.project.id=27f259ec3bd04186a70598926b09aa56, user.id=160079811a0e42009ddb10073ea2cd06) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:46:54.285925 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ff3ce7-0e86-44bb-9661-f6d5e9babaed None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:46:54.287460 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ff3ce7-0e86-44bb-9661-f6d5e9babaed None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:46:54.308685 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 51/96] 10.222.0.46 () {56 vars in 1374 bytes} [Thu Jun 23 15:46:53 2022] GET /identity/v3/role_assignments?user.id=160079811a0e42009ddb10073ea2cd06&scope.project.id=27f259ec3bd04186a70598926b09aa56&role.id=e2a5bbf3200a4d4091d72bc7ff78c34d => generated 259 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:46:54.356668 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:46:54.427017 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e079182e-c87f-4191-a2a7-d72315660cee None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:46:54.482571 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c555df7-ed6f-433a-8421-bc5eb2751439 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:46:54.483300 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c555df7-ed6f-433a-8421-bc5eb2751439 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:46:54.483594 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c555df7-ed6f-433a-8421-bc5eb2751439 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:46:54.483878 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c555df7-ed6f-433a-8421-bc5eb2751439 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:46:54.484754 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c555df7-ed6f-433a-8421-bc5eb2751439 None admin] RBAC: Adding query filter params (role.id=e2a5bbf3200a4d4091d72bc7ff78c34d, scope.project.id=34606c7f36fa455d8daf5ec64dafa1d4, user.id=160079811a0e42009ddb10073ea2cd06) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:46:54.485075 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c555df7-ed6f-433a-8421-bc5eb2751439 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:46:54.486749 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c555df7-ed6f-433a-8421-bc5eb2751439 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:46:54.498077 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 46/97] 10.222.0.46 () {56 vars in 1374 bytes} [Thu Jun 23 15:46:54 2022] GET /identity/v3/role_assignments?user.id=160079811a0e42009ddb10073ea2cd06&scope.project.id=34606c7f36fa455d8daf5ec64dafa1d4&role.id=e2a5bbf3200a4d4091d72bc7ff78c34d => generated 259 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:46:56.242805 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1538b15b-4bb5-49c1-a212-23ce216325b8 None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:46:56.243354 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1538b15b-4bb5-49c1-a212-23ce216325b8 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:46:56.243354 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1538b15b-4bb5-49c1-a212-23ce216325b8 None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:46:56.244788 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 52/98] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jun 23 15:46:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 15:46:56.250634 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83de5f04-42f0-4343-9aec-be2ba86d498a None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:46:56.251104 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83de5f04-42f0-4343-9aec-be2ba86d498a None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:46:56.251292 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83de5f04-42f0-4343-9aec-be2ba86d498a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:46:56.287728 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-83de5f04-42f0-4343-9aec-be2ba86d498a None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:46:56.288965 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-83de5f04-42f0-4343-9aec-be2ba86d498a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:46:56.373237 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 47/99] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jun 23 15:46:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 15:46:56.415706 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1538b15b-4bb5-49c1-a212-23ce216325b8 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:46:56.567277 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1538b15b-4bb5-49c1-a212-23ce216325b8 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:46:56.582211 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:46:56.605718 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49407cfe-2134-49c4-b2e6-d32709b69bf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:46:56.606879 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49407cfe-2134-49c4-b2e6-d32709b69bf8 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:46:56.607337 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49407cfe-2134-49c4-b2e6-d32709b69bf8 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:46:56.607780 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49407cfe-2134-49c4-b2e6-d32709b69bf8 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:46:56.619939 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49407cfe-2134-49c4-b2e6-d32709b69bf8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 15:46:56.628192 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-49407cfe-2134-49c4-b2e6-d32709b69bf8 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 23 15:46:56.628192 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:46:56.628192 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:46:56.628192 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:46:56.628192 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:46:56.628192 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:46:56.628192 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:46:56.628192 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:46:56.628192 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:46:56.628192 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:46:56.628192 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:46:56.628192 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:46:56.628192 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 15:46:56.628192 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 15:46:56.628192 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 15:46:56.628192 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 15:46:56.629022 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 15:46:56.629022 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 15:46:56.629022 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 15:46:56.629022 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 15:46:56.629022 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:46:56.629022 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:46:56.629022 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:46:56.629022 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:46:56.629022 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:46:56.629022 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:46:56.629022 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:46:56.629022 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Jun 23 15:46:56.629022 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:46:56.629022 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:46:56.629022 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:46:56.629022 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:46:56.629764 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:46:56.629764 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:46:56.629764 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:46:56.629764 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:46:56.629764 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 15:46:56.629764 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 15:46:56.629764 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 15:46:56.629764 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 15:46:56.629764 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 15:46:56.629764 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 15:46:56.629764 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 23 15:46:56.629764 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Jun 23 15:46:56.631747 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 53/100] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Jun 23 15:46:56 2022] GET /identity/v3/roles/reader => generated 84 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:46:56.639989 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83de5f04-42f0-4343-9aec-be2ba86d498a None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:46:56.648851 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c33231b-9003-4d5e-859b-e481d6896ba5 None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:46:56.649233 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c33231b-9003-4d5e-859b-e481d6896ba5 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:46:56.649422 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c33231b-9003-4d5e-859b-e481d6896ba5 None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:46:56.651022 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 54/101] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jun 23 15:46:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 15:46:56.656618 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010aa413-4c60-4e73-81a1-a162577d9cde None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:46:56.657023 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010aa413-4c60-4e73-81a1-a162577d9cde None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:46:56.657310 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010aa413-4c60-4e73-81a1-a162577d9cde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:46:56.687739 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-010aa413-4c60-4e73-81a1-a162577d9cde None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:46:56.688976 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-010aa413-4c60-4e73-81a1-a162577d9cde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:46:56.727066 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:46:56.801913 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 55/102] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jun 23 15:46:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 15:46:56.803705 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83de5f04-42f0-4343-9aec-be2ba86d498a None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:46:56.840562 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010aa413-4c60-4e73-81a1-a162577d9cde None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:46:56.864305 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cce4205-2afe-4c08-9e20-ec749f772ae5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:46:56.865007 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cce4205-2afe-4c08-9e20-ec749f772ae5 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:46:56.865299 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cce4205-2afe-4c08-9e20-ec749f772ae5 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:46:56.865581 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cce4205-2afe-4c08-9e20-ec749f772ae5 None admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:46:56.866354 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cce4205-2afe-4c08-9e20-ec749f772ae5 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:46:56.866668 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cce4205-2afe-4c08-9e20-ec749f772ae5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:46:56.868090 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cce4205-2afe-4c08-9e20-ec749f772ae5 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:46:56.882670 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 48/103] 10.222.0.46 () {56 vars in 1101 bytes} [Thu Jun 23 15:46:56 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:46:56.888476 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cce4205-2afe-4c08-9e20-ec749f772ae5 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:46:56.964291 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010aa413-4c60-4e73-81a1-a162577d9cde None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:46:56.987138 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:46:57.023228 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecd10a2-ca6d-4e98-a78b-076d7faec1c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:46:57.025594 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecd10a2-ca6d-4e98-a78b-076d7faec1c6 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:46:57.026373 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecd10a2-ca6d-4e98-a78b-076d7faec1c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:46:57.027641 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecd10a2-ca6d-4e98-a78b-076d7faec1c6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:46:57.030048 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fecd10a2-ca6d-4e98-a78b-076d7faec1c6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:46:57.033244 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fecd10a2-ca6d-4e98-a78b-076d7faec1c6 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:46:57.037241 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 56/104] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Jun 23 15:46:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:46:57.042190 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecd10a2-ca6d-4e98-a78b-076d7faec1c6 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:46:57.111209 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cce4205-2afe-4c08-9e20-ec749f772ae5 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:46:57.131097 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:46:57.153970 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecd10a2-ca6d-4e98-a78b-076d7faec1c6 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:46:57.191330 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33f7d6b-2fc8-49ec-be20-fcebafe63b3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:46:57.192025 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4897ba37-f758-44ab-8761-9141f2f4c528 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:46:57.192648 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33f7d6b-2fc8-49ec-be20-fcebafe63b3b None admin] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:46:57.192922 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4897ba37-f758-44ab-8761-9141f2f4c528 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:46:57.193216 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4897ba37-f758-44ab-8761-9141f2f4c528 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:46:57.193499 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4897ba37-f758-44ab-8761-9141f2f4c528 None admin] PATH_INFO: `/v3/users/233da9d863e245f0b59eef6794b16abd` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:46:57.193801 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33f7d6b-2fc8-49ec-be20-fcebafe63b3b None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:46:57.194320 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33f7d6b-2fc8-49ec-be20-fcebafe63b3b None admin] PATH_INFO: `/v3/projects` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:46:57.195317 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33f7d6b-2fc8-49ec-be20-fcebafe63b3b None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:46:57.197558 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33f7d6b-2fc8-49ec-be20-fcebafe63b3b None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:46:57.220578 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4897ba37-f758-44ab-8761-9141f2f4c528 None admin] RBAC: Authorizing `identity:get_user(user_id=233da9d863e245f0b59eef6794b16abd)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:46:57.222144 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4897ba37-f758-44ab-8761-9141f2f4c528 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:46:57.224004 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 49/105] 10.222.0.46 () {56 vars in 1177 bytes} [Thu Jun 23 15:46:56 2022] GET /identity/v3/users/233da9d863e245f0b59eef6794b16abd => generated 289 bytes in 336 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 15:46:57.229534 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4897ba37-f758-44ab-8761-9141f2f4c528 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:46:57.230818 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 57/106] 10.222.0.46 () {60 vars in 1141 bytes} [Thu Jun 23 15:46:57 2022] POST /identity/v3/projects => generated 302 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 15:46:57.313646 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b7ac26-62a8-46cc-afd7-4e43dc47889f None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:46:57.314835 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b7ac26-62a8-46cc-afd7-4e43dc47889f None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:46:57.315439 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b7ac26-62a8-46cc-afd7-4e43dc47889f None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:46:57.318677 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 58/107] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jun 23 15:46:57 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 15:46:57.324285 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f166873-83e0-4d87-8ab8-69014a5dc9ae None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:46:57.324928 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f166873-83e0-4d87-8ab8-69014a5dc9ae None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:46:57.325427 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f166873-83e0-4d87-8ab8-69014a5dc9ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:46:57.352033 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-7f166873-83e0-4d87-8ab8-69014a5dc9ae None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:46:57.354539 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-7f166873-83e0-4d87-8ab8-69014a5dc9ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:46:57.403254 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:46:57.450256 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4897ba37-f758-44ab-8761-9141f2f4c528 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:46:57.526767 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 59/108] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jun 23 15:46:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 15:46:57.529608 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4653e55-22d4-4755-b664-04d0e089d2af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:46:57.529608 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4653e55-22d4-4755-b664-04d0e089d2af None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:46:57.529608 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4653e55-22d4-4755-b664-04d0e089d2af None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:46:57.529878 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4653e55-22d4-4755-b664-04d0e089d2af None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:46:57.553653 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f166873-83e0-4d87-8ab8-69014a5dc9ae None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:46:57.589692 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:46:57.635226 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4653e55-22d4-4755-b664-04d0e089d2af None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:46:57.636724 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4653e55-22d4-4755-b664-04d0e089d2af None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:46:57.749876 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f166873-83e0-4d87-8ab8-69014a5dc9ae None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:46:57.790904 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 50/109] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Jun 23 15:46:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 562 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 15:46:57.792611 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd3e962-bea6-4268-ba89-40b5dec9ced2 None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:46:57.793127 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd3e962-bea6-4268-ba89-40b5dec9ced2 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:46:57.793617 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd3e962-bea6-4268-ba89-40b5dec9ced2 None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:46:57.795194 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 51/110] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jun 23 15:46:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 15:46:57.800357 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051e6048-79d5-447c-a7e1-5d7a066c3572 None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:46:57.800357 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051e6048-79d5-447c-a7e1-5d7a066c3572 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:46:57.800357 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051e6048-79d5-447c-a7e1-5d7a066c3572 None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:46:57.802259 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 52/111] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jun 23 15:46:57 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 15:46:57.803502 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051e6048-79d5-447c-a7e1-5d7a066c3572 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:46:57.810262 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0548c62-8313-454a-8d8e-5fef2d79b990 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:46:57.811173 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0548c62-8313-454a-8d8e-5fef2d79b990 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:46:57.811458 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0548c62-8313-454a-8d8e-5fef2d79b990 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:46:57.811737 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0548c62-8313-454a-8d8e-5fef2d79b990 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:46:57.826253 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0548c62-8313-454a-8d8e-5fef2d79b990 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 15:46:57.834163 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-b0548c62-8313-454a-8d8e-5fef2d79b990 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 15:46:57.834163 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:46:57.834163 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:46:57.834163 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:46:57.834163 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:46:57.834163 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:46:57.834163 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:46:57.834163 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:46:57.834163 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:46:57.834163 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:46:57.834163 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:46:57.834163 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:46:57.834163 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 15:46:57.834163 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 15:46:57.834163 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 15:46:57.834163 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 15:46:57.835097 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 15:46:57.835097 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 15:46:57.835097 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 15:46:57.835097 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 15:46:57.835097 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:46:57.835097 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:46:57.835097 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:46:57.835097 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:46:57.835097 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:46:57.835097 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:46:57.835097 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:46:57.835097 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Jun 23 15:46:57.835097 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:46:57.835097 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:46:57.835097 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:46:57.835097 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:46:57.835880 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:46:57.835880 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:46:57.835880 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:46:57.835880 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:46:57.835880 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 15:46:57.835880 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 15:46:57.835880 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 15:46:57.835880 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 15:46:57.835880 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 15:46:57.835880 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 15:46:57.835880 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 15:46:57.835880 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Jun 23 15:46:57.835880 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 60/112] 10.222.0.46 () {56 vars in 1096 bytes} [Thu Jun 23 15:46:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 15:46:57.840569 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e4b076-a73f-4efe-878b-a7be4ed65994 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:46:57.841201 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e4b076-a73f-4efe-878b-a7be4ed65994 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:46:57.841507 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e4b076-a73f-4efe-878b-a7be4ed65994 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:46:57.858982 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-58e4b076-a73f-4efe-878b-a7be4ed65994 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:46:57.860160 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-58e4b076-a73f-4efe-878b-a7be4ed65994 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:46:57.869397 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:46:57.882852 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:46:57.919638 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051e6048-79d5-447c-a7e1-5d7a066c3572 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:46:57.934558 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 61/113] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jun 23 15:46:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 15:46:57.935897 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d43876-4ec5-4d00-a78b-8a4c90a520b6 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:46:57.936189 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d43876-4ec5-4d00-a78b-8a4c90a520b6 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:46:57.936471 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d43876-4ec5-4d00-a78b-8a4c90a520b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:46:57.959389 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-73d43876-4ec5-4d00-a78b-8a4c90a520b6 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:46:57.960603 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-73d43876-4ec5-4d00-a78b-8a4c90a520b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:46:57.967338 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2209d76d-884c-4d45-acc7-5b9e3cb84b1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:46:57.968228 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2209d76d-884c-4d45-acc7-5b9e3cb84b1e None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:46:57.968517 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2209d76d-884c-4d45-acc7-5b9e3cb84b1e None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:46:57.969337 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2209d76d-884c-4d45-acc7-5b9e3cb84b1e None admin] PATH_INFO: `/v3/projects/27f259ec3bd04186a70598926b09aa56` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:46:57.970556 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2209d76d-884c-4d45-acc7-5b9e3cb84b1e None admin] RBAC: Authorizing `identity:get_project(project_id=27f259ec3bd04186a70598926b09aa56)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:46:57.972773 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2209d76d-884c-4d45-acc7-5b9e3cb84b1e None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:46:57.974716 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 53/114] 10.222.0.46 () {56 vars in 1186 bytes} [Thu Jun 23 15:46:57 2022] GET /identity/v3/projects/27f259ec3bd04186a70598926b09aa56 => generated 298 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:46:57.975961 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57795649-126c-4b90-8719-956b099edf38 None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:46:57.976254 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57795649-126c-4b90-8719-956b099edf38 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:46:57.976557 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57795649-126c-4b90-8719-956b099edf38 None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:46:57.978332 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 54/115] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jun 23 15:46:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 15:46:57.982916 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57795649-126c-4b90-8719-956b099edf38 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:46:58.037245 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 62/116] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jun 23 15:46:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 15:46:58.044077 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d43876-4ec5-4d00-a78b-8a4c90a520b6 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:46:58.092441 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57795649-126c-4b90-8719-956b099edf38 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:46:58.140107 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:46:58.141660 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38dac56-3921-43cc-a4ca-5e2c203e910c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:46:58.142499 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38dac56-3921-43cc-a4ca-5e2c203e910c None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:46:58.142964 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38dac56-3921-43cc-a4ca-5e2c203e910c None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:46:58.143379 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38dac56-3921-43cc-a4ca-5e2c203e910c None admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:46:58.144278 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38dac56-3921-43cc-a4ca-5e2c203e910c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:46:58.144750 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38dac56-3921-43cc-a4ca-5e2c203e910c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:46:58.147261 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38dac56-3921-43cc-a4ca-5e2c203e910c None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:46:58.151025 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:46:58.163422 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 55/117] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Jun 23 15:46:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:46:58.164004 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38dac56-3921-43cc-a4ca-5e2c203e910c None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:46:58.178130 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d43876-4ec5-4d00-a78b-8a4c90a520b6 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:46:58.217495 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24b06c0-a90d-45bc-8d0c-e43efe462d8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:46:58.218363 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24b06c0-a90d-45bc-8d0c-e43efe462d8b None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:46:58.218639 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24b06c0-a90d-45bc-8d0c-e43efe462d8b None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:46:58.218943 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24b06c0-a90d-45bc-8d0c-e43efe462d8b None admin] PATH_INFO: `/v3/roles/member` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:46:58.229909 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24b06c0-a90d-45bc-8d0c-e43efe462d8b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 15:46:58.237555 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-b24b06c0-a90d-45bc-8d0c-e43efe462d8b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 23 15:46:58.237555 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:46:58.237555 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:46:58.237555 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:46:58.237555 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:46:58.237555 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:46:58.237555 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:46:58.237555 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:46:58.237555 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:46:58.237555 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:46:58.237555 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:46:58.237555 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:46:58.237555 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 15:46:58.237555 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 15:46:58.237555 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 15:46:58.237555 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 15:46:58.238406 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 15:46:58.238406 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 15:46:58.238406 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 15:46:58.238406 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 15:46:58.238406 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:46:58.238406 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:46:58.238406 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:46:58.238406 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:46:58.238406 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:46:58.238406 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:46:58.238406 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:46:58.238406 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Jun 23 15:46:58.238406 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:46:58.238406 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:46:58.238406 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:46:58.238406 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:46:58.239343 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:46:58.239343 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:46:58.239343 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:46:58.239343 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:46:58.239343 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 15:46:58.239343 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 15:46:58.239343 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 15:46:58.239343 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 15:46:58.239343 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 15:46:58.239343 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 15:46:58.239343 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 23 15:46:58.239343 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Jun 23 15:46:58.239343 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 63/118] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Jun 23 15:46:58 2022] GET /identity/v3/roles/member => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:46:58.245310 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9884b0c4-b044-4af1-9a05-962ec366f1d2 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:46:58.245605 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9884b0c4-b044-4af1-9a05-962ec366f1d2 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:46:58.245889 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9884b0c4-b044-4af1-9a05-962ec366f1d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:46:58.264255 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-9884b0c4-b044-4af1-9a05-962ec366f1d2 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:46:58.265477 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-9884b0c4-b044-4af1-9a05-962ec366f1d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:46:58.274418 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38dac56-3921-43cc-a4ca-5e2c203e910c None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:46:58.312239 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b050308-1893-4298-8285-7c93e5b7ab74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:46:58.312868 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b050308-1893-4298-8285-7c93e5b7ab74 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:46:58.313112 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b050308-1893-4298-8285-7c93e5b7ab74 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:46:58.313391 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b050308-1893-4298-8285-7c93e5b7ab74 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:46:58.314269 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b050308-1893-4298-8285-7c93e5b7ab74 None admin] RBAC: Adding query filter params (role.id=38bbc1cebab84cac9b213ffba44bb4a6, scope.project.id=27f259ec3bd04186a70598926b09aa56, user.id=233da9d863e245f0b59eef6794b16abd) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:46:58.314536 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b050308-1893-4298-8285-7c93e5b7ab74 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:46:58.316111 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b050308-1893-4298-8285-7c93e5b7ab74 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:46:58.324244 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 56/119] 10.222.0.46 () {56 vars in 1374 bytes} [Thu Jun 23 15:46:58 2022] GET /identity/v3/role_assignments?user.id=233da9d863e245f0b59eef6794b16abd&scope.project.id=27f259ec3bd04186a70598926b09aa56&role.id=38bbc1cebab84cac9b213ffba44bb4a6 => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:46:58.335802 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b050308-1893-4298-8285-7c93e5b7ab74 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:46:58.338017 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 64/120] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jun 23 15:46:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 15:46:58.347786 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9884b0c4-b044-4af1-9a05-962ec366f1d2 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:46:58.422802 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:46:58.427616 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:46:58.476547 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9884b0c4-b044-4af1-9a05-962ec366f1d2 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:46:58.483906 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b050308-1893-4298-8285-7c93e5b7ab74 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:46:58.518534 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c6dd7e-9a29-4687-ab0c-100c4a87642c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:46:58.519690 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c6dd7e-9a29-4687-ab0c-100c4a87642c None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:46:58.520230 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c6dd7e-9a29-4687-ab0c-100c4a87642c None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:46:58.520609 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c6dd7e-9a29-4687-ab0c-100c4a87642c None admin] PATH_INFO: `/v3/users/e9c7fd1e1ef54284a2b594ce3d58fc5a` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:46:58.521934 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c6dd7e-9a29-4687-ab0c-100c4a87642c None admin] RBAC: Authorizing `identity:get_user(user_id=e9c7fd1e1ef54284a2b594ce3d58fc5a)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:46:58.523623 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c6dd7e-9a29-4687-ab0c-100c4a87642c None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:46:58.525504 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 65/121] 10.222.0.46 () {56 vars in 1177 bytes} [Thu Jun 23 15:46:58 2022] GET /identity/v3/users/e9c7fd1e1ef54284a2b594ce3d58fc5a => generated 247 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:46:58.527433 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b269086-c72e-4a1a-86a9-e9fecd0de86f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:46:58.528810 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b269086-c72e-4a1a-86a9-e9fecd0de86f None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:46:58.529407 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b269086-c72e-4a1a-86a9-e9fecd0de86f None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:46:58.529841 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b269086-c72e-4a1a-86a9-e9fecd0de86f None admin] PATH_INFO: `/v3/roles/member` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:46:58.530906 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c6dd7e-9a29-4687-ab0c-100c4a87642c None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:46:58.543257 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b269086-c72e-4a1a-86a9-e9fecd0de86f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 15:46:58.551235 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-1b269086-c72e-4a1a-86a9-e9fecd0de86f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 23 15:46:58.551235 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:46:58.551235 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:46:58.551235 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:46:58.551235 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:46:58.551235 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:46:58.551235 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:46:58.551235 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:46:58.551235 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:46:58.551235 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:46:58.551235 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:46:58.551235 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:46:58.551235 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 15:46:58.551235 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 15:46:58.551235 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 15:46:58.551235 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 15:46:58.552113 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 15:46:58.552113 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 15:46:58.552113 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 15:46:58.552113 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 15:46:58.552113 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:46:58.552113 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:46:58.552113 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:46:58.552113 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:46:58.552113 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:46:58.552113 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:46:58.552113 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:46:58.552113 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Jun 23 15:46:58.552113 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:46:58.552113 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:46:58.552113 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:46:58.552113 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:46:58.552881 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:46:58.552881 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:46:58.552881 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:46:58.552881 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:46:58.552881 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 15:46:58.552881 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 15:46:58.552881 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 15:46:58.552881 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 15:46:58.552881 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 15:46:58.552881 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 15:46:58.552881 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 23 15:46:58.552881 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Jun 23 15:46:58.557929 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 57/122] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Jun 23 15:46:58 2022] GET /identity/v3/roles/member => generated 84 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:46:58.558935 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b269086-c72e-4a1a-86a9-e9fecd0de86f None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:46:58.672451 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c6dd7e-9a29-4687-ab0c-100c4a87642c None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:46:58.679745 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b269086-c72e-4a1a-86a9-e9fecd0de86f None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:46:58.707009 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:46:58.721106 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:46:58.721305 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b82ee7-8b55-41f8-b556-1200f1799edd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:46:58.722157 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b82ee7-8b55-41f8-b556-1200f1799edd None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:46:58.722556 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b82ee7-8b55-41f8-b556-1200f1799edd None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:46:58.722881 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b82ee7-8b55-41f8-b556-1200f1799edd None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:46:58.731039 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1477a47-5899-464d-9707-0c23e0b9ea81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:46:58.731999 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1477a47-5899-464d-9707-0c23e0b9ea81 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:46:58.732454 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1477a47-5899-464d-9707-0c23e0b9ea81 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:46:58.732894 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1477a47-5899-464d-9707-0c23e0b9ea81 None admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:46:58.734260 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1477a47-5899-464d-9707-0c23e0b9ea81 None admin] RBAC: Adding query filter params (name=member) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:46:58.734783 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1477a47-5899-464d-9707-0c23e0b9ea81 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:46:58.737877 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1477a47-5899-464d-9707-0c23e0b9ea81 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:46:58.741929 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b82ee7-8b55-41f8-b556-1200f1799edd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 15:46:58.755755 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-49b82ee7-8b55-41f8-b556-1200f1799edd None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 15:46:58.755755 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:46:58.755755 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:46:58.755755 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:46:58.755755 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:46:58.755755 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:46:58.755755 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:46:58.755755 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:46:58.755755 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:46:58.755755 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:46:58.755755 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:46:58.755755 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:46:58.755755 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 15:46:58.755755 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 15:46:58.755755 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 15:46:58.755755 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 15:46:58.756738 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 15:46:58.756738 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 15:46:58.756738 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 15:46:58.756738 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 15:46:58.756738 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:46:58.756738 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:46:58.756738 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:46:58.756738 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:46:58.756738 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:46:58.756738 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:46:58.756738 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:46:58.756738 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Jun 23 15:46:58.756738 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:46:58.756738 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:46:58.756738 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:46:58.756738 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:46:58.757513 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:46:58.757513 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:46:58.757513 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:46:58.757513 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:46:58.757513 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 15:46:58.757513 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 15:46:58.757513 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 15:46:58.757513 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 15:46:58.757513 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 15:46:58.757513 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 15:46:58.757513 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 15:46:58.757513 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Jun 23 15:46:58.757513 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 58/124] 10.222.0.46 () {56 vars in 1114 bytes} [Thu Jun 23 15:46:58 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:46:58.757513 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b82ee7-8b55-41f8-b556-1200f1799edd None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:46:58.759011 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 66/124] 10.222.0.46 () {56 vars in 1101 bytes} [Thu Jun 23 15:46:58 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:46:58.760574 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1477a47-5899-464d-9707-0c23e0b9ea81 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:46:58.875466 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b82ee7-8b55-41f8-b556-1200f1799edd None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:46:58.913086 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f063d424-1e87-4a35-86cc-14b8f3b6818b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:46:58.913510 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1477a47-5899-464d-9707-0c23e0b9ea81 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:46:58.914031 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f063d424-1e87-4a35-86cc-14b8f3b6818b None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:46:58.914321 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f063d424-1e87-4a35-86cc-14b8f3b6818b None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:46:58.914603 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f063d424-1e87-4a35-86cc-14b8f3b6818b None admin] PATH_INFO: `/v3/projects/27f259ec3bd04186a70598926b09aa56` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:46:58.916661 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f063d424-1e87-4a35-86cc-14b8f3b6818b None admin] RBAC: Authorizing `identity:get_project(project_id=27f259ec3bd04186a70598926b09aa56)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:46:58.918455 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f063d424-1e87-4a35-86cc-14b8f3b6818b None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:46:58.920243 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 59/125] 10.222.0.46 () {56 vars in 1186 bytes} [Thu Jun 23 15:46:58 2022] GET /identity/v3/projects/27f259ec3bd04186a70598926b09aa56 => generated 298 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:46:58.921079 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f063d424-1e87-4a35-86cc-14b8f3b6818b None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:46:58.961214 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287f0da2-6a04-42f4-b75a-49ec394a3f43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:46:58.962829 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287f0da2-6a04-42f4-b75a-49ec394a3f43 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:46:58.963889 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287f0da2-6a04-42f4-b75a-49ec394a3f43 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:46:58.964418 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287f0da2-6a04-42f4-b75a-49ec394a3f43 None admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:46:58.965924 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287f0da2-6a04-42f4-b75a-49ec394a3f43 None admin] RBAC: Adding query filter params (name=member) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:46:58.966488 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287f0da2-6a04-42f4-b75a-49ec394a3f43 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:46:58.968221 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287f0da2-6a04-42f4-b75a-49ec394a3f43 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:46:58.979189 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:46:58.981677 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 67/126] 10.222.0.46 () {56 vars in 1101 bytes} [Thu Jun 23 15:46:58 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:46:58.982643 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287f0da2-6a04-42f4-b75a-49ec394a3f43 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:46:59.038008 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f063d424-1e87-4a35-86cc-14b8f3b6818b None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:46:59.055078 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:46:59.082238 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b478494b-16e0-4b7c-8770-eb1044d04879 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:46:59.082998 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b478494b-16e0-4b7c-8770-eb1044d04879 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:46:59.083294 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b478494b-16e0-4b7c-8770-eb1044d04879 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:46:59.083580 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b478494b-16e0-4b7c-8770-eb1044d04879 None admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:46:59.084348 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b478494b-16e0-4b7c-8770-eb1044d04879 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:46:59.084662 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b478494b-16e0-4b7c-8770-eb1044d04879 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:46:59.086464 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b478494b-16e0-4b7c-8770-eb1044d04879 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:46:59.099597 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 60/127] 10.222.0.46 () {56 vars in 1111 bytes} [Thu Jun 23 15:46:58 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:46:59.099597 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b478494b-16e0-4b7c-8770-eb1044d04879 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:46:59.124587 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287f0da2-6a04-42f4-b75a-49ec394a3f43 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:46:59.172799 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b91abad-0898-4895-b6a2-fd8bad89d4af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:46:59.174028 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b91abad-0898-4895-b6a2-fd8bad89d4af None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:46:59.174715 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b91abad-0898-4895-b6a2-fd8bad89d4af None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:46:59.175803 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b91abad-0898-4895-b6a2-fd8bad89d4af None admin] PATH_INFO: `/v3/users/160079811a0e42009ddb10073ea2cd06` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:46:59.177857 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b91abad-0898-4895-b6a2-fd8bad89d4af None admin] RBAC: Authorizing `identity:get_user(user_id=160079811a0e42009ddb10073ea2cd06)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:46:59.179698 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b91abad-0898-4895-b6a2-fd8bad89d4af None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:46:59.186756 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 68/128] 10.222.0.46 () {56 vars in 1177 bytes} [Thu Jun 23 15:46:58 2022] GET /identity/v3/users/160079811a0e42009ddb10073ea2cd06 => generated 275 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:46:59.186756 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b91abad-0898-4895-b6a2-fd8bad89d4af None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:46:59.237186 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b478494b-16e0-4b7c-8770-eb1044d04879 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:46:59.252635 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:46:59.278890 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532e1881-ce9c-4d32-8d40-6a5a1ba3028a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:46:59.280209 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532e1881-ce9c-4d32-8d40-6a5a1ba3028a None admin] REQUEST_METHOD: `PUT` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:46:59.280847 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532e1881-ce9c-4d32-8d40-6a5a1ba3028a None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:46:59.281418 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532e1881-ce9c-4d32-8d40-6a5a1ba3028a None admin] PATH_INFO: `/v3/projects/27f259ec3bd04186a70598926b09aa56/users/e9c7fd1e1ef54284a2b594ce3d58fc5a/roles/812d3c567c52469cb2b35709fbdc857c` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:46:59.283903 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532e1881-ce9c-4d32-8d40-6a5a1ba3028a None admin] RBAC: Authorizing `identity:create_grant(project_id=27f259ec3bd04186a70598926b09aa56, user_id=e9c7fd1e1ef54284a2b594ce3d58fc5a, role_id=812d3c567c52469cb2b35709fbdc857c)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:46:59.285583 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532e1881-ce9c-4d32-8d40-6a5a1ba3028a None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:46:59.305200 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 61/129] 10.222.0.46 () {58 vars in 1439 bytes} [Thu Jun 23 15:46:59 2022] PUT /identity/v3/projects/27f259ec3bd04186a70598926b09aa56/users/e9c7fd1e1ef54284a2b594ce3d58fc5a/roles/812d3c567c52469cb2b35709fbdc857c => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 15:46:59.307322 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532e1881-ce9c-4d32-8d40-6a5a1ba3028a None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:46:59.317291 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b91abad-0898-4895-b6a2-fd8bad89d4af None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:46:59.367250 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:46:59.395855 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5ad7e6-7bf6-42f5-a32a-a464432c4799 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:46:59.396871 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5ad7e6-7bf6-42f5-a32a-a464432c4799 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:46:59.397162 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5ad7e6-7bf6-42f5-a32a-a464432c4799 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:46:59.397455 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5ad7e6-7bf6-42f5-a32a-a464432c4799 None admin] PATH_INFO: `/v3/users/160079811a0e42009ddb10073ea2cd06` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:46:59.400118 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5ad7e6-7bf6-42f5-a32a-a464432c4799 None admin] RBAC: Authorizing `identity:get_user(user_id=160079811a0e42009ddb10073ea2cd06)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:46:59.401993 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5ad7e6-7bf6-42f5-a32a-a464432c4799 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:46:59.404223 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 69/130] 10.222.0.46 () {56 vars in 1177 bytes} [Thu Jun 23 15:46:59 2022] GET /identity/v3/users/160079811a0e42009ddb10073ea2cd06 => generated 275 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:46:59.405022 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5ad7e6-7bf6-42f5-a32a-a464432c4799 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:46:59.453858 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532e1881-ce9c-4d32-8d40-6a5a1ba3028a None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:46:59.516570 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfba787f-9a00-473e-baaf-1ba2e8ee1093 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:46:59.518444 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfba787f-9a00-473e-baaf-1ba2e8ee1093 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:46:59.519023 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfba787f-9a00-473e-baaf-1ba2e8ee1093 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:46:59.519517 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfba787f-9a00-473e-baaf-1ba2e8ee1093 None admin] PATH_INFO: `/v3/users/160079811a0e42009ddb10073ea2cd06` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:46:59.527730 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfba787f-9a00-473e-baaf-1ba2e8ee1093 None admin] RBAC: Authorizing `identity:get_user(user_id=160079811a0e42009ddb10073ea2cd06)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:46:59.529739 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfba787f-9a00-473e-baaf-1ba2e8ee1093 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:46:59.531947 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 62/131] 10.222.0.46 () {56 vars in 1177 bytes} [Thu Jun 23 15:46:59 2022] GET /identity/v3/users/160079811a0e42009ddb10073ea2cd06 => generated 275 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:46:59.533084 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfba787f-9a00-473e-baaf-1ba2e8ee1093 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:46:59.589964 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5ad7e6-7bf6-42f5-a32a-a464432c4799 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:46:59.615987 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:46:59.640680 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a9c0c2-21b4-4cfe-8301-da2350a84bec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:46:59.642019 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a9c0c2-21b4-4cfe-8301-da2350a84bec None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:46:59.642348 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a9c0c2-21b4-4cfe-8301-da2350a84bec None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:46:59.642632 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a9c0c2-21b4-4cfe-8301-da2350a84bec None admin] PATH_INFO: `/v3/projects/27f259ec3bd04186a70598926b09aa56` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:46:59.643929 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a9c0c2-21b4-4cfe-8301-da2350a84bec None admin] RBAC: Authorizing `identity:get_project(project_id=27f259ec3bd04186a70598926b09aa56)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:46:59.645773 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a9c0c2-21b4-4cfe-8301-da2350a84bec None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:46:59.647561 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 70/132] 10.222.0.46 () {56 vars in 1186 bytes} [Thu Jun 23 15:46:59 2022] GET /identity/v3/projects/27f259ec3bd04186a70598926b09aa56 => generated 298 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:46:59.648428 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a9c0c2-21b4-4cfe-8301-da2350a84bec None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:46:59.705348 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfba787f-9a00-473e-baaf-1ba2e8ee1093 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:46:59.720353 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:46:59.748081 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6790bbc5-8c5f-4c7f-a4ba-94d418b7d8ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:46:59.749351 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6790bbc5-8c5f-4c7f-a4ba-94d418b7d8ce None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:46:59.749642 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6790bbc5-8c5f-4c7f-a4ba-94d418b7d8ce None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:46:59.749925 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6790bbc5-8c5f-4c7f-a4ba-94d418b7d8ce None admin] PATH_INFO: `/v3/projects/34606c7f36fa455d8daf5ec64dafa1d4` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:46:59.751437 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6790bbc5-8c5f-4c7f-a4ba-94d418b7d8ce None admin] RBAC: Authorizing `identity:get_project(project_id=34606c7f36fa455d8daf5ec64dafa1d4)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:46:59.753084 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6790bbc5-8c5f-4c7f-a4ba-94d418b7d8ce None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:46:59.758803 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 63/133] 10.222.0.46 () {56 vars in 1186 bytes} [Thu Jun 23 15:46:59 2022] GET /identity/v3/projects/34606c7f36fa455d8daf5ec64dafa1d4 => generated 312 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:46:59.760798 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6790bbc5-8c5f-4c7f-a4ba-94d418b7d8ce None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:46:59.775590 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a9c0c2-21b4-4cfe-8301-da2350a84bec None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:46:59.812698 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1de5e06-d3b6-45e6-9ebf-9226311ce228 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:46:59.813960 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1de5e06-d3b6-45e6-9ebf-9226311ce228 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:46:59.814445 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1de5e06-d3b6-45e6-9ebf-9226311ce228 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:46:59.814949 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1de5e06-d3b6-45e6-9ebf-9226311ce228 None admin] PATH_INFO: `/v3/projects/27f259ec3bd04186a70598926b09aa56` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:46:59.816435 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1de5e06-d3b6-45e6-9ebf-9226311ce228 None admin] RBAC: Authorizing `identity:get_project(project_id=27f259ec3bd04186a70598926b09aa56)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:46:59.818135 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1de5e06-d3b6-45e6-9ebf-9226311ce228 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:46:59.820185 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 71/134] 10.222.0.46 () {56 vars in 1186 bytes} [Thu Jun 23 15:46:59 2022] GET /identity/v3/projects/27f259ec3bd04186a70598926b09aa56 => generated 298 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:46:59.822362 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9ebb3e-e9e0-4ece-bca2-1148ed8e5268 None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:46:59.822928 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9ebb3e-e9e0-4ece-bca2-1148ed8e5268 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:46:59.823422 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9ebb3e-e9e0-4ece-bca2-1148ed8e5268 None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:46:59.824923 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 72/135] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jun 23 15:46:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 15:46:59.825866 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9ebb3e-e9e0-4ece-bca2-1148ed8e5268 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:46:59.910879 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6790bbc5-8c5f-4c7f-a4ba-94d418b7d8ce None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:46:59.943127 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:46:59.947941 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18027307-1035-4777-b312-698ceb0bcb08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:46:59.949215 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18027307-1035-4777-b312-698ceb0bcb08 None admin] REQUEST_METHOD: `PUT` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:46:59.949729 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18027307-1035-4777-b312-698ceb0bcb08 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:46:59.950283 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18027307-1035-4777-b312-698ceb0bcb08 None admin] PATH_INFO: `/v3/projects/27f259ec3bd04186a70598926b09aa56/users/160079811a0e42009ddb10073ea2cd06/roles/e2a5bbf3200a4d4091d72bc7ff78c34d` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:46:59.952886 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18027307-1035-4777-b312-698ceb0bcb08 None admin] RBAC: Authorizing `identity:create_grant(project_id=27f259ec3bd04186a70598926b09aa56, user_id=160079811a0e42009ddb10073ea2cd06, role_id=e2a5bbf3200a4d4091d72bc7ff78c34d)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:46:59.953339 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9ebb3e-e9e0-4ece-bca2-1148ed8e5268 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:46:59.954901 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18027307-1035-4777-b312-698ceb0bcb08 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:46:59.971971 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 64/136] 10.222.0.46 () {58 vars in 1439 bytes} [Thu Jun 23 15:46:59 2022] PUT /identity/v3/projects/27f259ec3bd04186a70598926b09aa56/users/160079811a0e42009ddb10073ea2cd06/roles/e2a5bbf3200a4d4091d72bc7ff78c34d => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 15:46:59.974719 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18027307-1035-4777-b312-698ceb0bcb08 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:46:59.984246 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:46:59.995680 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3066df4c-7ba9-4682-970c-13480335254d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:46:59.996805 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3066df4c-7ba9-4682-970c-13480335254d None admin] REQUEST_METHOD: `PUT` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:46:59.996942 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3066df4c-7ba9-4682-970c-13480335254d None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:46:59.997161 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3066df4c-7ba9-4682-970c-13480335254d None admin] PATH_INFO: `/v3/projects/34606c7f36fa455d8daf5ec64dafa1d4/users/160079811a0e42009ddb10073ea2cd06/roles/e2a5bbf3200a4d4091d72bc7ff78c34d` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:46:59.999469 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3066df4c-7ba9-4682-970c-13480335254d None admin] RBAC: Authorizing `identity:create_grant(project_id=34606c7f36fa455d8daf5ec64dafa1d4, user_id=160079811a0e42009ddb10073ea2cd06, role_id=e2a5bbf3200a4d4091d72bc7ff78c34d)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:00.001025 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3066df4c-7ba9-4682-970c-13480335254d None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:00.049398 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 73/137] 10.222.0.46 () {58 vars in 1439 bytes} [Thu Jun 23 15:46:59 2022] PUT /identity/v3/projects/34606c7f36fa455d8daf5ec64dafa1d4/users/160079811a0e42009ddb10073ea2cd06/roles/e2a5bbf3200a4d4091d72bc7ff78c34d => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 15:47:00.050749 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58e1402-a7f0-4f85-98f4-28eab5b7d07b None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:00.051048 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58e1402-a7f0-4f85-98f4-28eab5b7d07b None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:00.051330 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58e1402-a7f0-4f85-98f4-28eab5b7d07b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:00.098818 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-e58e1402-a7f0-4f85-98f4-28eab5b7d07b None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:47:00.100020 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-e58e1402-a7f0-4f85-98f4-28eab5b7d07b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:47:00.124395 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18027307-1035-4777-b312-698ceb0bcb08 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:00.177714 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d362731a-12a1-4d12-8807-9e75fb4dc321 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:00.179085 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d362731a-12a1-4d12-8807-9e75fb4dc321 None admin] REQUEST_METHOD: `PUT` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:00.179606 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d362731a-12a1-4d12-8807-9e75fb4dc321 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:00.180091 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d362731a-12a1-4d12-8807-9e75fb4dc321 None admin] PATH_INFO: `/v3/projects/27f259ec3bd04186a70598926b09aa56/users/160079811a0e42009ddb10073ea2cd06/roles/d4d557a3ec4e434f8e83499c5fcc7136` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:00.183977 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d362731a-12a1-4d12-8807-9e75fb4dc321 None admin] RBAC: Authorizing `identity:create_grant(project_id=27f259ec3bd04186a70598926b09aa56, user_id=160079811a0e42009ddb10073ea2cd06, role_id=d4d557a3ec4e434f8e83499c5fcc7136)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:00.186161 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d362731a-12a1-4d12-8807-9e75fb4dc321 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:00.204601 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 65/138] 10.222.0.46 () {58 vars in 1439 bytes} [Thu Jun 23 15:46:59 2022] PUT /identity/v3/projects/27f259ec3bd04186a70598926b09aa56/users/160079811a0e42009ddb10073ea2cd06/roles/d4d557a3ec4e434f8e83499c5fcc7136 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 15:47:00.283632 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 74/139] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jun 23 15:47:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 15:47:00.310722 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d362731a-12a1-4d12-8807-9e75fb4dc321 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:00.348695 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:00.449014 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d362731a-12a1-4d12-8807-9e75fb4dc321 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:00.493855 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8086adbd-ff5b-437a-a7a7-96b8a9c8a521 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:00.494880 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8086adbd-ff5b-437a-a7a7-96b8a9c8a521 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:00.495238 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8086adbd-ff5b-437a-a7a7-96b8a9c8a521 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:00.495589 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8086adbd-ff5b-437a-a7a7-96b8a9c8a521 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:00.499587 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8086adbd-ff5b-437a-a7a7-96b8a9c8a521 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:00.501150 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8086adbd-ff5b-437a-a7a7-96b8a9c8a521 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:00.503261 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 66/140] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Jun 23 15:47:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:47:00.508681 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58e1402-a7f0-4f85-98f4-28eab5b7d07b None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:00.684608 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58e1402-a7f0-4f85-98f4-28eab5b7d07b None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:00.714077 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:00.743512 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bf2665-077a-4e71-91e8-c0d8666b7dfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:00.744234 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bf2665-077a-4e71-91e8-c0d8666b7dfb None admin] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:00.744528 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bf2665-077a-4e71-91e8-c0d8666b7dfb None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:00.744812 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bf2665-077a-4e71-91e8-c0d8666b7dfb None admin] PATH_INFO: `/v3/users` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:00.745616 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3bf2665-077a-4e71-91e8-c0d8666b7dfb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:00.747164 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3bf2665-077a-4e71-91e8-c0d8666b7dfb None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:00.780721 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 75/141] 10.222.0.46 () {60 vars in 1132 bytes} [Thu Jun 23 15:47:00 2022] POST /identity/v3/users => generated 283 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 15:47:00.962923 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d203fa-96d3-4dd6-8513-70c8fae34897 None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:00.963475 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d203fa-96d3-4dd6-8513-70c8fae34897 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:00.963550 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d203fa-96d3-4dd6-8513-70c8fae34897 None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:00.965126 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 67/142] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jun 23 15:47:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 15:47:00.970900 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbc445a-5956-489a-b6c2-602767157f2a None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:00.971199 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbc445a-5956-489a-b6c2-602767157f2a None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:00.971488 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbc445a-5956-489a-b6c2-602767157f2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:00.996652 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-1bbc445a-5956-489a-b6c2-602767157f2a None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:47:00.997882 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-1bbc445a-5956-489a-b6c2-602767157f2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:47:01.101536 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 76/143] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jun 23 15:47:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 15:47:01.139287 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d203fa-96d3-4dd6-8513-70c8fae34897 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:01.281814 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:01.290386 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d203fa-96d3-4dd6-8513-70c8fae34897 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:01.333055 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54195539-a976-49ff-be63-a5becfdd2339 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:01.333871 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54195539-a976-49ff-be63-a5becfdd2339 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:01.334160 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54195539-a976-49ff-be63-a5becfdd2339 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:01.334438 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54195539-a976-49ff-be63-a5becfdd2339 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:01.347994 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54195539-a976-49ff-be63-a5becfdd2339 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 15:47:01.356118 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-54195539-a976-49ff-be63-a5becfdd2339 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 23 15:47:01.356118 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:47:01.356118 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:47:01.356118 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:47:01.356118 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:47:01.356118 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:47:01.356118 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:47:01.356118 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:47:01.356118 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:47:01.356118 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:47:01.356118 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:47:01.356118 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:47:01.356118 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 15:47:01.356118 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 15:47:01.356118 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 15:47:01.356118 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 15:47:01.356973 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 15:47:01.356973 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 15:47:01.356973 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 15:47:01.356973 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 15:47:01.356973 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:47:01.356973 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:47:01.356973 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:47:01.356973 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:47:01.356973 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:47:01.356973 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:47:01.356973 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:47:01.356973 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Jun 23 15:47:01.356973 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:47:01.356973 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:47:01.356973 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:47:01.356973 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:47:01.358157 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:47:01.358157 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:47:01.358157 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:47:01.358157 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:47:01.358157 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 15:47:01.358157 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 15:47:01.358157 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 15:47:01.358157 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 15:47:01.358157 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 15:47:01.358157 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 15:47:01.358157 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 23 15:47:01.358157 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Jun 23 15:47:01.358157 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 68/144] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Jun 23 15:47:01 2022] GET /identity/v3/roles/reader => generated 84 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:47:01.363578 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbc445a-5956-489a-b6c2-602767157f2a None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:01.475960 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbc445a-5956-489a-b6c2-602767157f2a None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:01.478123 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:01.515389 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862e0417-c187-453b-818e-18f161433f43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:01.516158 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862e0417-c187-453b-818e-18f161433f43 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:01.516342 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862e0417-c187-453b-818e-18f161433f43 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:01.516668 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862e0417-c187-453b-818e-18f161433f43 None admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:01.517434 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862e0417-c187-453b-818e-18f161433f43 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:47:01.517781 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862e0417-c187-453b-818e-18f161433f43 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:01.519349 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862e0417-c187-453b-818e-18f161433f43 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:01.528980 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 77/145] 10.222.0.46 () {56 vars in 1101 bytes} [Thu Jun 23 15:47:01 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 15:47:01.534779 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54195539-a976-49ff-be63-a5becfdd2339 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:01.664116 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54195539-a976-49ff-be63-a5becfdd2339 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:01.701896 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e415174f-cdfe-4690-93c8-7ae1b6485e6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:01.702867 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e415174f-cdfe-4690-93c8-7ae1b6485e6f None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:01.703150 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e415174f-cdfe-4690-93c8-7ae1b6485e6f None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:01.703423 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e415174f-cdfe-4690-93c8-7ae1b6485e6f None admin] PATH_INFO: `/v3/users/233da9d863e245f0b59eef6794b16abd` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:01.704852 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e415174f-cdfe-4690-93c8-7ae1b6485e6f None admin] RBAC: Authorizing `identity:get_user(user_id=233da9d863e245f0b59eef6794b16abd)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:01.706633 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e415174f-cdfe-4690-93c8-7ae1b6485e6f None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:01.708686 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 69/146] 10.222.0.46 () {56 vars in 1177 bytes} [Thu Jun 23 15:47:01 2022] GET /identity/v3/users/233da9d863e245f0b59eef6794b16abd => generated 289 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:47:01.714809 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862e0417-c187-453b-818e-18f161433f43 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:01.836594 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862e0417-c187-453b-818e-18f161433f43 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:01.874145 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c1fa39-2eff-4d5f-9aae-f9a8165a65ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:01.875063 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c1fa39-2eff-4d5f-9aae-f9a8165a65ab None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:01.875345 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c1fa39-2eff-4d5f-9aae-f9a8165a65ab None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:01.875616 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c1fa39-2eff-4d5f-9aae-f9a8165a65ab None admin] PATH_INFO: `/v3/projects/27f259ec3bd04186a70598926b09aa56` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:01.876920 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c1fa39-2eff-4d5f-9aae-f9a8165a65ab None admin] RBAC: Authorizing `identity:get_project(project_id=27f259ec3bd04186a70598926b09aa56)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:01.878396 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c1fa39-2eff-4d5f-9aae-f9a8165a65ab None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:01.880321 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 78/147] 10.222.0.46 () {56 vars in 1186 bytes} [Thu Jun 23 15:47:01 2022] GET /identity/v3/projects/27f259ec3bd04186a70598926b09aa56 => generated 298 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:47:01.886132 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e415174f-cdfe-4690-93c8-7ae1b6485e6f None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:01.919781 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:02.003260 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e415174f-cdfe-4690-93c8-7ae1b6485e6f None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:02.039920 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb412b7-e542-471d-addf-570f47b89a53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:02.041421 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb412b7-e542-471d-addf-570f47b89a53 None admin] REQUEST_METHOD: `PUT` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:02.041706 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb412b7-e542-471d-addf-570f47b89a53 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:02.041981 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb412b7-e542-471d-addf-570f47b89a53 None admin] PATH_INFO: `/v3/projects/27f259ec3bd04186a70598926b09aa56/users/233da9d863e245f0b59eef6794b16abd/roles/38bbc1cebab84cac9b213ffba44bb4a6` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:02.044076 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb412b7-e542-471d-addf-570f47b89a53 None admin] RBAC: Authorizing `identity:create_grant(project_id=27f259ec3bd04186a70598926b09aa56, user_id=233da9d863e245f0b59eef6794b16abd, role_id=38bbc1cebab84cac9b213ffba44bb4a6)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:02.045541 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb412b7-e542-471d-addf-570f47b89a53 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:02.069645 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 70/148] 10.222.0.46 () {58 vars in 1439 bytes} [Thu Jun 23 15:47:01 2022] PUT /identity/v3/projects/27f259ec3bd04186a70598926b09aa56/users/233da9d863e245f0b59eef6794b16abd/roles/38bbc1cebab84cac9b213ffba44bb4a6 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 15:47:02.227586 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7747c6bb-aed0-4964-b0cf-6ded407cd473 None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:02.227763 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7747c6bb-aed0-4964-b0cf-6ded407cd473 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:02.228043 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7747c6bb-aed0-4964-b0cf-6ded407cd473 None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:02.229727 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 79/149] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jun 23 15:47:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 15:47:02.235638 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7336ed4c-1b27-46ff-9f8a-4b0d94556df9 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:02.235941 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7336ed4c-1b27-46ff-9f8a-4b0d94556df9 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:02.236218 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7336ed4c-1b27-46ff-9f8a-4b0d94556df9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:02.258222 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-7336ed4c-1b27-46ff-9f8a-4b0d94556df9 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:47:02.260225 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-7336ed4c-1b27-46ff-9f8a-4b0d94556df9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:47:02.348693 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 71/150] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jun 23 15:47:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 15:47:02.375904 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7747c6bb-aed0-4964-b0cf-6ded407cd473 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:02.429339 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:02.545123 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7747c6bb-aed0-4964-b0cf-6ded407cd473 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:02.601421 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a944f5-dd46-4c32-8e50-018ce94f90c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:02.602290 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a944f5-dd46-4c32-8e50-018ce94f90c2 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:02.602654 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a944f5-dd46-4c32-8e50-018ce94f90c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:02.602907 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a944f5-dd46-4c32-8e50-018ce94f90c2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:02.615024 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a944f5-dd46-4c32-8e50-018ce94f90c2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 15:47:02.631865 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-f1a944f5-dd46-4c32-8e50-018ce94f90c2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 15:47:02.631865 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:47:02.631865 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:47:02.631865 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:47:02.631865 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:47:02.631865 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:47:02.631865 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:47:02.631865 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:47:02.631865 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:47:02.631865 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:47:02.631865 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:47:02.631865 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:47:02.631865 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 15:47:02.631865 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 15:47:02.631865 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 15:47:02.631865 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 15:47:02.632857 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 15:47:02.632857 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 15:47:02.632857 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 15:47:02.632857 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 15:47:02.632857 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:47:02.632857 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:47:02.632857 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:47:02.632857 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:47:02.632857 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:47:02.632857 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:47:02.632857 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:47:02.632857 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Jun 23 15:47:02.632857 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:47:02.632857 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:47:02.632857 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:47:02.632857 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:47:02.634679 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:47:02.634679 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:47:02.634679 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:47:02.634679 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:47:02.634679 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 15:47:02.634679 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 15:47:02.634679 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 15:47:02.634679 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 15:47:02.634679 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 15:47:02.634679 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 15:47:02.634679 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 15:47:02.634679 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Jun 23 15:47:02.634679 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 80/151] 10.222.0.46 () {56 vars in 1096 bytes} [Thu Jun 23 15:47:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:47:02.640566 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7336ed4c-1b27-46ff-9f8a-4b0d94556df9 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:02.657752 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:02.824452 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7336ed4c-1b27-46ff-9f8a-4b0d94556df9 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:02.872750 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f068be8d-0124-47d1-a2db-d83824e0abf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:02.873451 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f068be8d-0124-47d1-a2db-d83824e0abf7 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:02.874019 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f068be8d-0124-47d1-a2db-d83824e0abf7 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:02.874307 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f068be8d-0124-47d1-a2db-d83824e0abf7 None admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:02.875114 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f068be8d-0124-47d1-a2db-d83824e0abf7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:47:02.875434 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f068be8d-0124-47d1-a2db-d83824e0abf7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:02.876845 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f068be8d-0124-47d1-a2db-d83824e0abf7 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:02.891625 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 72/152] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Jun 23 15:47:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:47:02.901053 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a944f5-dd46-4c32-8e50-018ce94f90c2 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:03.046373 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:03.055965 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a944f5-dd46-4c32-8e50-018ce94f90c2 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:03.099581 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f706fde-2ea7-4e00-ad3d-40ee04352c83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:03.100480 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f706fde-2ea7-4e00-ad3d-40ee04352c83 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:03.100762 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f706fde-2ea7-4e00-ad3d-40ee04352c83 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:03.101147 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f706fde-2ea7-4e00-ad3d-40ee04352c83 None admin] PATH_INFO: `/v3/users/e9c7fd1e1ef54284a2b594ce3d58fc5a` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:03.102284 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f706fde-2ea7-4e00-ad3d-40ee04352c83 None admin] RBAC: Authorizing `identity:get_user(user_id=e9c7fd1e1ef54284a2b594ce3d58fc5a)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:03.103823 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f706fde-2ea7-4e00-ad3d-40ee04352c83 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:03.105848 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 81/153] 10.222.0.46 () {56 vars in 1177 bytes} [Thu Jun 23 15:47:02 2022] GET /identity/v3/users/e9c7fd1e1ef54284a2b594ce3d58fc5a => generated 247 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:47:03.111425 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f068be8d-0124-47d1-a2db-d83824e0abf7 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:03.261582 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:03.292837 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f068be8d-0124-47d1-a2db-d83824e0abf7 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:03.335831 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ecd683-d476-473c-a555-1b27e6567a02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:03.336509 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ecd683-d476-473c-a555-1b27e6567a02 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:03.336833 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ecd683-d476-473c-a555-1b27e6567a02 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:03.337113 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ecd683-d476-473c-a555-1b27e6567a02 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:03.427876 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ecd683-d476-473c-a555-1b27e6567a02 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:03.429294 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ecd683-d476-473c-a555-1b27e6567a02 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:03.516917 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:03.555465 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 73/154] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Jun 23 15:47:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 445 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 15:47:03.562475 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f706fde-2ea7-4e00-ad3d-40ee04352c83 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:03.627720 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41612ea8-e861-4892-9854-af0d4c8bfa5d None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:03.628424 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41612ea8-e861-4892-9854-af0d4c8bfa5d None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:03.628836 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41612ea8-e861-4892-9854-af0d4c8bfa5d None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:03.636054 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 74/155] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jun 23 15:47:03 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 15:47:03.644204 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45aadbb-d322-49f4-8b53-20b10c40fee3 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:03.644499 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45aadbb-d322-49f4-8b53-20b10c40fee3 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:03.644784 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45aadbb-d322-49f4-8b53-20b10c40fee3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:03.671028 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-c45aadbb-d322-49f4-8b53-20b10c40fee3 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:47:03.672275 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-c45aadbb-d322-49f4-8b53-20b10c40fee3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:47:03.783737 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 75/156] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jun 23 15:47:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 15:47:03.785050 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31685ca-708d-487a-ad2d-6768a374ba60 None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:03.785791 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31685ca-708d-487a-ad2d-6768a374ba60 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:03.786079 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31685ca-708d-487a-ad2d-6768a374ba60 None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:03.787488 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 76/157] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jun 23 15:47:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 15:47:03.793501 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37c677d-6085-4e75-b3ed-ecfe2fd84667 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:03.793793 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37c677d-6085-4e75-b3ed-ecfe2fd84667 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:03.794072 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37c677d-6085-4e75-b3ed-ecfe2fd84667 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:03.817879 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-f37c677d-6085-4e75-b3ed-ecfe2fd84667 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:47:03.819408 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-f37c677d-6085-4e75-b3ed-ecfe2fd84667 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:47:03.880229 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f706fde-2ea7-4e00-ad3d-40ee04352c83 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:04.001549 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 77/158] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jun 23 15:47:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 15:47:04.003235 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37c677d-6085-4e75-b3ed-ecfe2fd84667 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:04.021268 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68ab7f8-e0d3-4ebe-8079-88af5b1f47fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:04.021268 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68ab7f8-e0d3-4ebe-8079-88af5b1f47fd None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:04.021268 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68ab7f8-e0d3-4ebe-8079-88af5b1f47fd None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:04.021268 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68ab7f8-e0d3-4ebe-8079-88af5b1f47fd None admin] PATH_INFO: `/v3/projects/27f259ec3bd04186a70598926b09aa56` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:04.027540 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68ab7f8-e0d3-4ebe-8079-88af5b1f47fd None admin] RBAC: Authorizing `identity:get_project(project_id=27f259ec3bd04186a70598926b09aa56)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:04.029358 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68ab7f8-e0d3-4ebe-8079-88af5b1f47fd None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:04.031387 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 82/159] 10.222.0.46 () {56 vars in 1186 bytes} [Thu Jun 23 15:47:03 2022] GET /identity/v3/projects/27f259ec3bd04186a70598926b09aa56 => generated 298 bytes in 470 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:47:04.039481 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68ab7f8-e0d3-4ebe-8079-88af5b1f47fd None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:04.070581 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:04.127076 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:04.272225 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68ab7f8-e0d3-4ebe-8079-88af5b1f47fd None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:04.294813 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37c677d-6085-4e75-b3ed-ecfe2fd84667 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:04.322349 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716b105a-982a-4bad-8760-55394cc2c74f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:04.323061 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716b105a-982a-4bad-8760-55394cc2c74f None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:04.323349 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716b105a-982a-4bad-8760-55394cc2c74f None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:04.323630 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716b105a-982a-4bad-8760-55394cc2c74f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:04.324496 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716b105a-982a-4bad-8760-55394cc2c74f None admin] RBAC: Adding query filter params (role.id=812d3c567c52469cb2b35709fbdc857c, scope.project.id=27f259ec3bd04186a70598926b09aa56, user.id=e9c7fd1e1ef54284a2b594ce3d58fc5a) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:47:04.324816 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716b105a-982a-4bad-8760-55394cc2c74f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:04.326333 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716b105a-982a-4bad-8760-55394cc2c74f None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:04.336390 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 83/160] 10.222.0.46 () {56 vars in 1374 bytes} [Thu Jun 23 15:47:04 2022] GET /identity/v3/role_assignments?user.id=e9c7fd1e1ef54284a2b594ce3d58fc5a&scope.project.id=27f259ec3bd04186a70598926b09aa56&role.id=812d3c567c52469cb2b35709fbdc857c => generated 608 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 15:47:04.338136 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716b105a-982a-4bad-8760-55394cc2c74f None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:04.373604 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2696ab-448c-453e-9a68-cbb6b6eabd18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:04.374507 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2696ab-448c-453e-9a68-cbb6b6eabd18 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:04.374887 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2696ab-448c-453e-9a68-cbb6b6eabd18 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:04.375391 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2696ab-448c-453e-9a68-cbb6b6eabd18 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:04.403450 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2696ab-448c-453e-9a68-cbb6b6eabd18 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 15:47:04.417554 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-6d2696ab-448c-453e-9a68-cbb6b6eabd18 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 23 15:47:04.417554 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:47:04.417554 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:47:04.417554 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:47:04.417554 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:47:04.417554 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:47:04.417554 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:47:04.417554 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:47:04.417554 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:47:04.417554 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:47:04.417554 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:47:04.417554 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:47:04.417554 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 15:47:04.417554 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 15:47:04.417554 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 15:47:04.417554 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 15:47:04.418488 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 15:47:04.418488 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 15:47:04.418488 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 15:47:04.418488 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 15:47:04.418488 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:47:04.418488 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:47:04.418488 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:47:04.418488 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:47:04.418488 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:47:04.418488 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:47:04.418488 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:47:04.418488 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Jun 23 15:47:04.418488 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:47:04.418488 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:47:04.418488 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:47:04.418488 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:47:04.419294 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:47:04.419294 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:47:04.419294 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:47:04.419294 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:47:04.419294 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 15:47:04.419294 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 15:47:04.419294 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 15:47:04.419294 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 15:47:04.419294 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 15:47:04.419294 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 15:47:04.419294 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 23 15:47:04.419294 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Jun 23 15:47:04.420342 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 78/161] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Jun 23 15:47:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 418 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:47:04.421672 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7feb7706-f6f4-472c-9fe9-80bca3c1a139 None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:04.422241 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7feb7706-f6f4-472c-9fe9-80bca3c1a139 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:04.422771 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7feb7706-f6f4-472c-9fe9-80bca3c1a139 None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:04.424478 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 79/162] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jun 23 15:47:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 15:47:04.436372 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b666e345-3aff-4729-a954-b3344cb5cd49 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:04.437032 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b666e345-3aff-4729-a954-b3344cb5cd49 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:04.437524 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b666e345-3aff-4729-a954-b3344cb5cd49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:04.466989 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-b666e345-3aff-4729-a954-b3344cb5cd49 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:47:04.468230 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-b666e345-3aff-4729-a954-b3344cb5cd49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:47:04.495424 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:04.508410 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716b105a-982a-4bad-8760-55394cc2c74f None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:04.585921 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0a2aab-f788-4964-8990-458e574afcd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:04.586816 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0a2aab-f788-4964-8990-458e574afcd2 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:04.587618 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0a2aab-f788-4964-8990-458e574afcd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:04.587900 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0a2aab-f788-4964-8990-458e574afcd2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:04.602466 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0a2aab-f788-4964-8990-458e574afcd2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 15:47:04.611766 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-2b0a2aab-f788-4964-8990-458e574afcd2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 23 15:47:04.611766 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:47:04.611766 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:47:04.611766 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:47:04.611766 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:47:04.611766 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:47:04.611766 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:47:04.611766 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:47:04.611766 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:47:04.611766 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:47:04.611766 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:47:04.611766 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:47:04.611766 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 15:47:04.611766 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 15:47:04.611766 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 15:47:04.611766 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 15:47:04.612733 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 15:47:04.612733 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 15:47:04.612733 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 15:47:04.612733 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 15:47:04.612733 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:47:04.612733 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:47:04.612733 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:47:04.612733 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:47:04.612733 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:47:04.612733 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:47:04.612733 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:47:04.612733 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Jun 23 15:47:04.612733 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:47:04.612733 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:47:04.612733 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:47:04.612733 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:47:04.613501 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:47:04.613501 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:47:04.613501 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:47:04.613501 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:47:04.613501 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 15:47:04.613501 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 15:47:04.613501 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 15:47:04.613501 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 15:47:04.613501 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 15:47:04.613501 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 15:47:04.613501 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 23 15:47:04.613501 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Jun 23 15:47:04.616566 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 84/163] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Jun 23 15:47:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:47:04.618224 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0a2aab-f788-4964-8990-458e574afcd2 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:04.644114 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 80/164] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jun 23 15:47:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 15:47:04.645115 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b666e345-3aff-4729-a954-b3344cb5cd49 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:04.708711 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:04.747600 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0a2aab-f788-4964-8990-458e574afcd2 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:04.791878 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cedc78d-5efb-4218-a755-7506cf25180f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:04.792920 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cedc78d-5efb-4218-a755-7506cf25180f None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:04.794050 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cedc78d-5efb-4218-a755-7506cf25180f None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:04.794594 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cedc78d-5efb-4218-a755-7506cf25180f None admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:04.795633 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cedc78d-5efb-4218-a755-7506cf25180f None admin] RBAC: Adding query filter params (name=member) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:47:04.796170 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cedc78d-5efb-4218-a755-7506cf25180f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:04.797969 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cedc78d-5efb-4218-a755-7506cf25180f None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:04.810609 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 85/165] 10.222.0.46 () {56 vars in 1101 bytes} [Thu Jun 23 15:47:04 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:47:04.811963 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cedc78d-5efb-4218-a755-7506cf25180f None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:04.834876 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:04.870274 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b666e345-3aff-4729-a954-b3344cb5cd49 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:04.939492 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be0861a-a2ca-4fe7-be82-28064d473f0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:04.940692 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be0861a-a2ca-4fe7-be82-28064d473f0d None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:04.941176 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be0861a-a2ca-4fe7-be82-28064d473f0d None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:04.941654 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be0861a-a2ca-4fe7-be82-28064d473f0d None admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:04.943119 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be0861a-a2ca-4fe7-be82-28064d473f0d None admin] RBAC: Adding query filter params (name=member) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:47:04.943657 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be0861a-a2ca-4fe7-be82-28064d473f0d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:04.948944 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be0861a-a2ca-4fe7-be82-28064d473f0d None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:04.976327 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 81/166] 10.222.0.46 () {56 vars in 1101 bytes} [Thu Jun 23 15:47:04 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 332 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:47:04.977293 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be0861a-a2ca-4fe7-be82-28064d473f0d None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:05.007203 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cedc78d-5efb-4218-a755-7506cf25180f None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:05.066996 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93efae8-fa96-4f24-9511-9b09a6155cbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:05.067818 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93efae8-fa96-4f24-9511-9b09a6155cbf None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:05.068100 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93efae8-fa96-4f24-9511-9b09a6155cbf None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:05.068548 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93efae8-fa96-4f24-9511-9b09a6155cbf None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:05.091691 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93efae8-fa96-4f24-9511-9b09a6155cbf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 15:47:05.096386 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:05.106628 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-c93efae8-fa96-4f24-9511-9b09a6155cbf None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 15:47:05.106628 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:47:05.106628 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:47:05.106628 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:47:05.106628 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:47:05.106628 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:47:05.106628 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:47:05.106628 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:47:05.106628 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:47:05.106628 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:47:05.106628 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:47:05.106628 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:47:05.106628 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 15:47:05.106628 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 15:47:05.106628 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 15:47:05.106628 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 15:47:05.107609 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 15:47:05.107609 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 15:47:05.107609 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 15:47:05.107609 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 15:47:05.107609 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:47:05.107609 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:47:05.107609 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:47:05.107609 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:47:05.107609 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:47:05.107609 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:47:05.107609 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:47:05.107609 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Jun 23 15:47:05.107609 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:47:05.107609 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:47:05.107609 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:47:05.107609 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:47:05.108408 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:47:05.108408 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:47:05.108408 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:47:05.108408 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:47:05.108408 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 15:47:05.108408 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 15:47:05.108408 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 15:47:05.108408 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 15:47:05.108408 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 15:47:05.108408 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 15:47:05.108408 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 15:47:05.108408 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Jun 23 15:47:05.108408 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 86/167] 10.222.0.46 () {56 vars in 1114 bytes} [Thu Jun 23 15:47:04 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:47:05.109142 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93efae8-fa96-4f24-9511-9b09a6155cbf None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:05.112485 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be0861a-a2ca-4fe7-be82-28064d473f0d None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:05.172316 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5575094d-0c49-4c4d-83a7-80ff2bdd1aaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:05.173544 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5575094d-0c49-4c4d-83a7-80ff2bdd1aaa None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:05.173976 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5575094d-0c49-4c4d-83a7-80ff2bdd1aaa None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:05.174812 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5575094d-0c49-4c4d-83a7-80ff2bdd1aaa None admin] PATH_INFO: `/v3/users/160079811a0e42009ddb10073ea2cd06` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:05.177272 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5575094d-0c49-4c4d-83a7-80ff2bdd1aaa None admin] RBAC: Authorizing `identity:get_user(user_id=160079811a0e42009ddb10073ea2cd06)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:05.183981 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5575094d-0c49-4c4d-83a7-80ff2bdd1aaa None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:05.185975 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 82/168] 10.222.0.46 () {56 vars in 1177 bytes} [Thu Jun 23 15:47:04 2022] GET /identity/v3/users/160079811a0e42009ddb10073ea2cd06 => generated 275 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:47:05.186941 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5575094d-0c49-4c4d-83a7-80ff2bdd1aaa None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:05.286255 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:05.347938 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93efae8-fa96-4f24-9511-9b09a6155cbf None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:05.436902 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626cb6a8-949a-4a54-bd3d-c8a73d931f03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:05.437797 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626cb6a8-949a-4a54-bd3d-c8a73d931f03 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:05.438083 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626cb6a8-949a-4a54-bd3d-c8a73d931f03 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:05.438351 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626cb6a8-949a-4a54-bd3d-c8a73d931f03 None admin] PATH_INFO: `/v3/users/160079811a0e42009ddb10073ea2cd06` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:05.439770 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626cb6a8-949a-4a54-bd3d-c8a73d931f03 None admin] RBAC: Authorizing `identity:get_user(user_id=160079811a0e42009ddb10073ea2cd06)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:05.441244 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626cb6a8-949a-4a54-bd3d-c8a73d931f03 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:05.446274 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5575094d-0c49-4c4d-83a7-80ff2bdd1aaa None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:05.448179 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 87/169] 10.222.0.46 () {56 vars in 1177 bytes} [Thu Jun 23 15:47:05 2022] GET /identity/v3/users/160079811a0e42009ddb10073ea2cd06 => generated 275 bytes in 339 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:47:05.449025 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626cb6a8-949a-4a54-bd3d-c8a73d931f03 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:05.532811 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea03938-53e7-43aa-a513-5dd979e8d953 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:05.533508 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea03938-53e7-43aa-a513-5dd979e8d953 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:05.533799 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea03938-53e7-43aa-a513-5dd979e8d953 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:05.534080 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea03938-53e7-43aa-a513-5dd979e8d953 None admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:05.534885 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea03938-53e7-43aa-a513-5dd979e8d953 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:47:05.535263 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea03938-53e7-43aa-a513-5dd979e8d953 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:05.537358 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea03938-53e7-43aa-a513-5dd979e8d953 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:05.560847 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 83/170] 10.222.0.46 () {56 vars in 1111 bytes} [Thu Jun 23 15:47:05 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 374 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:47:05.561736 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea03938-53e7-43aa-a513-5dd979e8d953 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:05.599295 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:05.695084 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626cb6a8-949a-4a54-bd3d-c8a73d931f03 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:05.765637 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea03938-53e7-43aa-a513-5dd979e8d953 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:05.767697 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed30b9b-34c4-48e2-a5f3-ec8ea6a52ac5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:05.768368 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed30b9b-34c4-48e2-a5f3-ec8ea6a52ac5 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:05.768709 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed30b9b-34c4-48e2-a5f3-ec8ea6a52ac5 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:05.769043 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed30b9b-34c4-48e2-a5f3-ec8ea6a52ac5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:05.792907 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:05.822532 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2e34f5-aec2-45d3-9f57-d444844b11fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:05.823280 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2e34f5-aec2-45d3-9f57-d444844b11fb None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:05.823578 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2e34f5-aec2-45d3-9f57-d444844b11fb None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:05.823911 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2e34f5-aec2-45d3-9f57-d444844b11fb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:05.932778 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed30b9b-34c4-48e2-a5f3-ec8ea6a52ac5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:05.936068 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed30b9b-34c4-48e2-a5f3-ec8ea6a52ac5 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:05.936339 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2e34f5-aec2-45d3-9f57-d444844b11fb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:05.937828 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2e34f5-aec2-45d3-9f57-d444844b11fb None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:05.956832 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:06.051721 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 84/171] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Jun 23 15:47:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 490 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 15:47:06.052838 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2e34f5-aec2-45d3-9f57-d444844b11fb None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:06.072405 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 88/172] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Jun 23 15:47:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 624 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 15:47:06.074171 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af69ca56-346d-45db-900b-ee91aaeb69f3 None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:06.074605 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af69ca56-346d-45db-900b-ee91aaeb69f3 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:06.075043 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af69ca56-346d-45db-900b-ee91aaeb69f3 None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:06.076688 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 89/173] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jun 23 15:47:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 15:47:06.077977 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252250eb-8a89-4e27-b1b2-45a4afc893b3 None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:06.078470 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252250eb-8a89-4e27-b1b2-45a4afc893b3 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:06.078914 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252250eb-8a89-4e27-b1b2-45a4afc893b3 None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:06.080552 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 90/174] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jun 23 15:47:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 15:47:06.085656 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252250eb-8a89-4e27-b1b2-45a4afc893b3 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:06.132519 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:06.160920 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2e34f5-aec2-45d3-9f57-d444844b11fb None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:06.178896 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:06.198849 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac89e15-60e9-4785-871d-b1b4df658b4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:06.199073 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252250eb-8a89-4e27-b1b2-45a4afc893b3 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:06.199647 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac89e15-60e9-4785-871d-b1b4df658b4e None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:06.199939 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac89e15-60e9-4785-871d-b1b4df658b4e None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:06.200224 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac89e15-60e9-4785-871d-b1b4df658b4e None admin] PATH_INFO: `/v3/users/160079811a0e42009ddb10073ea2cd06` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:06.201738 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac89e15-60e9-4785-871d-b1b4df658b4e None admin] RBAC: Authorizing `identity:get_user(user_id=160079811a0e42009ddb10073ea2cd06)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:06.203443 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac89e15-60e9-4785-871d-b1b4df658b4e None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:06.205375 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 85/175] 10.222.0.46 () {56 vars in 1177 bytes} [Thu Jun 23 15:47:06 2022] GET /identity/v3/users/160079811a0e42009ddb10073ea2cd06 => generated 275 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:47:06.205903 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac89e15-60e9-4785-871d-b1b4df658b4e None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:06.251883 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0289b6-5054-4c72-b910-a04d16ea5e54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:06.253289 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0289b6-5054-4c72-b910-a04d16ea5e54 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:06.253694 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0289b6-5054-4c72-b910-a04d16ea5e54 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:06.254154 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0289b6-5054-4c72-b910-a04d16ea5e54 None admin] PATH_INFO: `/v3/projects/34606c7f36fa455d8daf5ec64dafa1d4` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:06.256163 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0289b6-5054-4c72-b910-a04d16ea5e54 None admin] RBAC: Authorizing `identity:get_project(project_id=34606c7f36fa455d8daf5ec64dafa1d4)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:06.258429 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0289b6-5054-4c72-b910-a04d16ea5e54 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:06.261434 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 91/176] 10.222.0.46 () {56 vars in 1186 bytes} [Thu Jun 23 15:47:06 2022] GET /identity/v3/projects/34606c7f36fa455d8daf5ec64dafa1d4 => generated 312 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:47:06.263418 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ebc399-8779-42db-953b-38a59266f47d None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:06.263893 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ebc399-8779-42db-953b-38a59266f47d None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:06.264418 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ebc399-8779-42db-953b-38a59266f47d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:06.289241 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-e7ebc399-8779-42db-953b-38a59266f47d None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:47:06.291745 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-e7ebc399-8779-42db-953b-38a59266f47d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:47:06.363033 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac89e15-60e9-4785-871d-b1b4df658b4e None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:06.389331 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 92/177] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jun 23 15:47:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 15:47:06.391623 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e288a1ca-351f-46e8-a115-61b13f29a095 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:06.391904 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e288a1ca-351f-46e8-a115-61b13f29a095 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:06.392190 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e288a1ca-351f-46e8-a115-61b13f29a095 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:06.397811 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9320d69-6f24-4a01-ad5f-eca50f2e554f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:06.398702 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9320d69-6f24-4a01-ad5f-eca50f2e554f None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:06.398996 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9320d69-6f24-4a01-ad5f-eca50f2e554f None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:06.399785 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9320d69-6f24-4a01-ad5f-eca50f2e554f None admin] PATH_INFO: `/v3/projects/27f259ec3bd04186a70598926b09aa56` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:06.400989 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9320d69-6f24-4a01-ad5f-eca50f2e554f None admin] RBAC: Authorizing `identity:get_project(project_id=27f259ec3bd04186a70598926b09aa56)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:06.402732 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9320d69-6f24-4a01-ad5f-eca50f2e554f None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:06.404736 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 86/178] 10.222.0.46 () {56 vars in 1186 bytes} [Thu Jun 23 15:47:06 2022] GET /identity/v3/projects/27f259ec3bd04186a70598926b09aa56 => generated 298 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:47:06.405772 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe5e9f2-9024-41df-b70e-a3174be98c8d None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:06.406061 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe5e9f2-9024-41df-b70e-a3174be98c8d None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:06.406342 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe5e9f2-9024-41df-b70e-a3174be98c8d None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:06.407734 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 87/179] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jun 23 15:47:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 15:47:06.408322 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe5e9f2-9024-41df-b70e-a3174be98c8d None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:06.420453 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-e288a1ca-351f-46e8-a115-61b13f29a095 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:47:06.422846 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-e288a1ca-351f-46e8-a115-61b13f29a095 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:47:06.459654 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:06.473254 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:06.514711 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 93/180] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jun 23 15:47:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 15:47:06.514711 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e288a1ca-351f-46e8-a115-61b13f29a095 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:06.539503 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe5e9f2-9024-41df-b70e-a3174be98c8d None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:06.577661 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1732b8d7-28a3-44e2-8341-dec397804eab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:06.578339 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1732b8d7-28a3-44e2-8341-dec397804eab None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:06.578628 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1732b8d7-28a3-44e2-8341-dec397804eab None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:06.578942 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1732b8d7-28a3-44e2-8341-dec397804eab None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:06.626819 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e288a1ca-351f-46e8-a115-61b13f29a095 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:06.650044 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1732b8d7-28a3-44e2-8341-dec397804eab None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:06.651807 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1732b8d7-28a3-44e2-8341-dec397804eab None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:06.663540 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f6d411-f954-4e65-9006-95d5d429060c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:06.664204 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f6d411-f954-4e65-9006-95d5d429060c None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:06.664490 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f6d411-f954-4e65-9006-95d5d429060c None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:06.664798 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f6d411-f954-4e65-9006-95d5d429060c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:06.665755 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f6d411-f954-4e65-9006-95d5d429060c None admin] RBAC: Adding query filter params (role.id=e2a5bbf3200a4d4091d72bc7ff78c34d, scope.project.id=34606c7f36fa455d8daf5ec64dafa1d4, user.id=160079811a0e42009ddb10073ea2cd06) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:47:06.665968 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f6d411-f954-4e65-9006-95d5d429060c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:06.667463 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f6d411-f954-4e65-9006-95d5d429060c None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:06.673572 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 94/181] 10.222.0.46 () {56 vars in 1374 bytes} [Thu Jun 23 15:47:06 2022] GET /identity/v3/role_assignments?user.id=160079811a0e42009ddb10073ea2cd06&scope.project.id=34606c7f36fa455d8daf5ec64dafa1d4&role.id=e2a5bbf3200a4d4091d72bc7ff78c34d => generated 608 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:47:06.674854 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13493f3-deb4-4aed-bb70-4953950694b7 None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:06.675145 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13493f3-deb4-4aed-bb70-4953950694b7 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:06.675423 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13493f3-deb4-4aed-bb70-4953950694b7 None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:06.676683 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 95/182] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jun 23 15:47:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 15:47:06.679560 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13493f3-deb4-4aed-bb70-4953950694b7 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:06.713218 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:06.728269 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:06.768876 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 88/183] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Jun 23 15:47:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 15:47:06.769776 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1732b8d7-28a3-44e2-8341-dec397804eab None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:06.811795 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13493f3-deb4-4aed-bb70-4953950694b7 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:06.856030 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bbe929-7e55-4291-9d72-aed563733b22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:06.856709 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bbe929-7e55-4291-9d72-aed563733b22 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:06.856993 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bbe929-7e55-4291-9d72-aed563733b22 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:06.857275 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bbe929-7e55-4291-9d72-aed563733b22 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:06.858497 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bbe929-7e55-4291-9d72-aed563733b22 None admin] RBAC: Adding query filter params (role.id=e2a5bbf3200a4d4091d72bc7ff78c34d, scope.project.id=27f259ec3bd04186a70598926b09aa56, user.id=160079811a0e42009ddb10073ea2cd06) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:47:06.858835 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bbe929-7e55-4291-9d72-aed563733b22 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:06.860639 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bbe929-7e55-4291-9d72-aed563733b22 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:06.867366 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 96/184] 10.222.0.46 () {56 vars in 1374 bytes} [Thu Jun 23 15:47:06 2022] GET /identity/v3/role_assignments?user.id=160079811a0e42009ddb10073ea2cd06&scope.project.id=27f259ec3bd04186a70598926b09aa56&role.id=e2a5bbf3200a4d4091d72bc7ff78c34d => generated 608 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:47:06.868425 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8947bb-0790-4290-a97c-c55dd992fdf5 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:06.868709 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8947bb-0790-4290-a97c-c55dd992fdf5 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:06.868984 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8947bb-0790-4290-a97c-c55dd992fdf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:06.886930 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1732b8d7-28a3-44e2-8341-dec397804eab None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:06.890179 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-8b8947bb-0790-4290-a97c-c55dd992fdf5 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:47:06.891659 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-8b8947bb-0790-4290-a97c-c55dd992fdf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:47:06.945644 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c8f697-697d-44f3-9d30-01573d1f2ad0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:06.948106 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c8f697-697d-44f3-9d30-01573d1f2ad0 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:06.948397 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c8f697-697d-44f3-9d30-01573d1f2ad0 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:06.948679 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c8f697-697d-44f3-9d30-01573d1f2ad0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:06.962802 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c8f697-697d-44f3-9d30-01573d1f2ad0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 15:47:06.971404 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-b6c8f697-697d-44f3-9d30-01573d1f2ad0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 15:47:06.971404 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:47:06.971404 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:47:06.971404 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:47:06.971404 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:47:06.971404 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:47:06.971404 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:47:06.971404 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:47:06.971404 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:47:06.971404 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:47:06.971404 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:47:06.971404 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:47:06.971404 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 15:47:06.971404 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 15:47:06.971404 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 15:47:06.971404 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 15:47:06.972494 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 15:47:06.972494 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 15:47:06.972494 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 15:47:06.972494 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 15:47:06.972494 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:47:06.972494 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:47:06.972494 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:47:06.972494 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:47:06.972494 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:47:06.972494 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:47:06.972494 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:47:06.972494 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Jun 23 15:47:06.972494 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:47:06.972494 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:47:06.972494 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:47:06.972494 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:47:06.973277 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:47:06.973277 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:47:06.973277 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:47:06.973277 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:47:06.973277 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 15:47:06.973277 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 15:47:06.973277 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 15:47:06.973277 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 15:47:06.973277 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 15:47:06.973277 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 15:47:06.973277 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 15:47:06.973277 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Jun 23 15:47:06.973277 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 89/185] 10.222.0.46 () {56 vars in 1096 bytes} [Thu Jun 23 15:47:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:47:06.973933 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c8f697-697d-44f3-9d30-01573d1f2ad0 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:06.990833 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 97/186] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jun 23 15:47:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 15:47:06.992600 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cfa6e3-3046-4c17-bcb3-67eef308f613 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:06.993491 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cfa6e3-3046-4c17-bcb3-67eef308f613 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:06.993840 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cfa6e3-3046-4c17-bcb3-67eef308f613 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:07.001538 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:07.012777 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-b3cfa6e3-3046-4c17-bcb3-67eef308f613 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:47:07.014127 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-b3cfa6e3-3046-4c17-bcb3-67eef308f613 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:47:07.041555 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:07.099768 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 98/187] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jun 23 15:47:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 15:47:07.100639 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cfa6e3-3046-4c17-bcb3-67eef308f613 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:07.114011 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c8f697-697d-44f3-9d30-01573d1f2ad0 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:07.149671 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81155d9e-7476-4047-b2c6-91e04ea3cdc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:07.150475 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81155d9e-7476-4047-b2c6-91e04ea3cdc5 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:07.150795 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81155d9e-7476-4047-b2c6-91e04ea3cdc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:07.151082 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81155d9e-7476-4047-b2c6-91e04ea3cdc5 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:07.161392 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81155d9e-7476-4047-b2c6-91e04ea3cdc5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 15:47:07.169147 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-81155d9e-7476-4047-b2c6-91e04ea3cdc5 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 23 15:47:07.169147 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:47:07.169147 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:47:07.169147 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:47:07.169147 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:47:07.169147 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:47:07.169147 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:47:07.169147 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:47:07.169147 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:47:07.169147 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:47:07.169147 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:47:07.169147 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:47:07.169147 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 15:47:07.169147 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 15:47:07.169147 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 15:47:07.169147 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 15:47:07.170054 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 15:47:07.170054 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 15:47:07.170054 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 15:47:07.170054 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 15:47:07.170054 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:47:07.170054 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:47:07.170054 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:47:07.170054 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:47:07.170054 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:47:07.170054 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:47:07.170054 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:47:07.170054 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Jun 23 15:47:07.170054 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:47:07.170054 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:47:07.170054 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:47:07.170054 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:47:07.170954 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:47:07.170954 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:47:07.170954 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:47:07.170954 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:47:07.170954 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 15:47:07.170954 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 15:47:07.170954 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 15:47:07.170954 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 15:47:07.170954 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 15:47:07.170954 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 15:47:07.170954 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 23 15:47:07.170954 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Jun 23 15:47:07.174985 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 90/188] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Jun 23 15:47:06 2022] GET /identity/v3/roles/reader => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:47:07.175744 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81155d9e-7476-4047-b2c6-91e04ea3cdc5 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:07.204715 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cfa6e3-3046-4c17-bcb3-67eef308f613 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:07.238803 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa50bd6b-ce11-488c-b29a-9d8e5d40b9b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:07.239598 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa50bd6b-ce11-488c-b29a-9d8e5d40b9b8 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:07.239886 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa50bd6b-ce11-488c-b29a-9d8e5d40b9b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:07.240189 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa50bd6b-ce11-488c-b29a-9d8e5d40b9b8 None admin] PATH_INFO: `/v3/projects/27f259ec3bd04186a70598926b09aa56` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:07.241600 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa50bd6b-ce11-488c-b29a-9d8e5d40b9b8 None admin] RBAC: Authorizing `identity:get_project(project_id=27f259ec3bd04186a70598926b09aa56)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:07.243852 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa50bd6b-ce11-488c-b29a-9d8e5d40b9b8 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:07.245709 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 99/189] 10.222.0.46 () {56 vars in 1186 bytes} [Thu Jun 23 15:47:07 2022] GET /identity/v3/projects/27f259ec3bd04186a70598926b09aa56 => generated 298 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:47:07.246479 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa50bd6b-ce11-488c-b29a-9d8e5d40b9b8 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:07.257952 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:07.274568 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:07.284491 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81155d9e-7476-4047-b2c6-91e04ea3cdc5 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:07.318848 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501e8059-b8f8-4366-84cb-e5a9d55cd89d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:07.319524 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501e8059-b8f8-4366-84cb-e5a9d55cd89d None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:07.319808 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501e8059-b8f8-4366-84cb-e5a9d55cd89d None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:07.320080 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501e8059-b8f8-4366-84cb-e5a9d55cd89d None admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:07.320830 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501e8059-b8f8-4366-84cb-e5a9d55cd89d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:47:07.321679 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501e8059-b8f8-4366-84cb-e5a9d55cd89d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:07.323064 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501e8059-b8f8-4366-84cb-e5a9d55cd89d None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:07.331442 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 91/190] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Jun 23 15:47:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:47:07.331442 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501e8059-b8f8-4366-84cb-e5a9d55cd89d None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:07.349322 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa50bd6b-ce11-488c-b29a-9d8e5d40b9b8 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:07.385746 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491f6292-8c91-4fee-b03e-bbd8318b1980 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:07.386596 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491f6292-8c91-4fee-b03e-bbd8318b1980 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:07.386960 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491f6292-8c91-4fee-b03e-bbd8318b1980 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:07.387238 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491f6292-8c91-4fee-b03e-bbd8318b1980 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:07.400374 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491f6292-8c91-4fee-b03e-bbd8318b1980 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 15:47:07.415739 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-491f6292-8c91-4fee-b03e-bbd8318b1980 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 15:47:07.415739 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:47:07.415739 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:47:07.415739 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:47:07.415739 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:47:07.415739 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:47:07.415739 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:47:07.415739 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:47:07.415739 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:47:07.415739 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:47:07.415739 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:47:07.415739 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:47:07.415739 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 15:47:07.415739 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 15:47:07.415739 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 15:47:07.415739 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 15:47:07.416637 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 15:47:07.416637 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 15:47:07.416637 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 15:47:07.416637 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 15:47:07.416637 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:47:07.416637 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:47:07.416637 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:47:07.416637 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:47:07.416637 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:47:07.416637 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:47:07.416637 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:47:07.416637 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Jun 23 15:47:07.416637 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:47:07.416637 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:47:07.416637 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:47:07.416637 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:47:07.417431 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:47:07.417431 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:47:07.417431 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:47:07.417431 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:47:07.417431 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 15:47:07.417431 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 15:47:07.417431 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 15:47:07.417431 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 15:47:07.417431 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 15:47:07.417431 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 15:47:07.417431 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 15:47:07.417431 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Jun 23 15:47:07.418015 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 100/191] 10.222.0.46 () {56 vars in 1114 bytes} [Thu Jun 23 15:47:07 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:47:07.418993 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491f6292-8c91-4fee-b03e-bbd8318b1980 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:07.445655 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501e8059-b8f8-4366-84cb-e5a9d55cd89d None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:07.480419 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94cd15c-46c6-41f0-8a4e-7e44b4f518d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:07.481511 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94cd15c-46c6-41f0-8a4e-7e44b4f518d2 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:07.481805 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94cd15c-46c6-41f0-8a4e-7e44b4f518d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:07.482090 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94cd15c-46c6-41f0-8a4e-7e44b4f518d2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:07.483310 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94cd15c-46c6-41f0-8a4e-7e44b4f518d2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:07.484761 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94cd15c-46c6-41f0-8a4e-7e44b4f518d2 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:07.486554 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 92/192] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Jun 23 15:47:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:47:07.487274 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94cd15c-46c6-41f0-8a4e-7e44b4f518d2 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:07.490756 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:07.510722 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:07.527238 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491f6292-8c91-4fee-b03e-bbd8318b1980 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:07.563352 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd16290-b816-433a-a979-b38e6a91108d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:07.564113 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd16290-b816-433a-a979-b38e6a91108d None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:07.564308 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd16290-b816-433a-a979-b38e6a91108d None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:07.564592 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd16290-b816-433a-a979-b38e6a91108d None admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:07.565344 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd16290-b816-433a-a979-b38e6a91108d None admin] RBAC: Adding query filter params (name=reader) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:47:07.565651 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd16290-b816-433a-a979-b38e6a91108d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:07.567084 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd16290-b816-433a-a979-b38e6a91108d None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:07.577242 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 101/193] 10.222.0.46 () {56 vars in 1101 bytes} [Thu Jun 23 15:47:07 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:47:07.578110 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd16290-b816-433a-a979-b38e6a91108d None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:07.593876 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94cd15c-46c6-41f0-8a4e-7e44b4f518d2 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:07.628355 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363c1f4e-633f-4a93-b4f3-702bd67ab508 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:07.629073 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363c1f4e-633f-4a93-b4f3-702bd67ab508 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:07.629299 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363c1f4e-633f-4a93-b4f3-702bd67ab508 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:07.629576 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363c1f4e-633f-4a93-b4f3-702bd67ab508 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:07.630414 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363c1f4e-633f-4a93-b4f3-702bd67ab508 None admin] RBAC: Adding query filter params (role.id=d4d557a3ec4e434f8e83499c5fcc7136, scope.project.id=27f259ec3bd04186a70598926b09aa56, user.id=160079811a0e42009ddb10073ea2cd06) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:47:07.630752 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363c1f4e-633f-4a93-b4f3-702bd67ab508 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:07.632214 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363c1f4e-633f-4a93-b4f3-702bd67ab508 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:07.638078 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 93/194] 10.222.0.46 () {56 vars in 1374 bytes} [Thu Jun 23 15:47:07 2022] GET /identity/v3/role_assignments?user.id=160079811a0e42009ddb10073ea2cd06&scope.project.id=27f259ec3bd04186a70598926b09aa56&role.id=d4d557a3ec4e434f8e83499c5fcc7136 => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:47:07.638903 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363c1f4e-633f-4a93-b4f3-702bd67ab508 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:07.695735 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd16290-b816-433a-a979-b38e6a91108d None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:07.722902 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:07.734277 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1c0ee8-3aa7-41e7-80b9-7919227620f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:07.735205 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1c0ee8-3aa7-41e7-80b9-7919227620f9 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:07.735490 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1c0ee8-3aa7-41e7-80b9-7919227620f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:07.735767 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1c0ee8-3aa7-41e7-80b9-7919227620f9 None admin] PATH_INFO: `/v3/users/f084629de0a54e1493d2a025da535a8d` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:07.752374 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1c0ee8-3aa7-41e7-80b9-7919227620f9 None admin] RBAC: Authorizing `identity:get_user(user_id=f084629de0a54e1493d2a025da535a8d)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:07.753816 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1c0ee8-3aa7-41e7-80b9-7919227620f9 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:07.755660 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 102/195] 10.222.0.46 () {56 vars in 1177 bytes} [Thu Jun 23 15:47:07 2022] GET /identity/v3/users/f084629de0a54e1493d2a025da535a8d => generated 283 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:47:07.756450 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1c0ee8-3aa7-41e7-80b9-7919227620f9 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:07.767027 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:07.775935 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363c1f4e-633f-4a93-b4f3-702bd67ab508 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:07.815941 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0ac8ca-d835-4f97-9b62-69ca20b81b9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:07.816629 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0ac8ca-d835-4f97-9b62-69ca20b81b9d None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:07.816922 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0ac8ca-d835-4f97-9b62-69ca20b81b9d None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:07.817202 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0ac8ca-d835-4f97-9b62-69ca20b81b9d None admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:07.817959 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0ac8ca-d835-4f97-9b62-69ca20b81b9d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:47:07.818269 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0ac8ca-d835-4f97-9b62-69ca20b81b9d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:07.819708 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0ac8ca-d835-4f97-9b62-69ca20b81b9d None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:07.828826 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 94/196] 10.222.0.46 () {56 vars in 1111 bytes} [Thu Jun 23 15:47:07 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:47:07.829732 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0ac8ca-d835-4f97-9b62-69ca20b81b9d None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:07.877625 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1c0ee8-3aa7-41e7-80b9-7919227620f9 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:07.912606 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347e1543-5865-4117-b72a-0c90aa4228af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:07.913278 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347e1543-5865-4117-b72a-0c90aa4228af None admin] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:07.913562 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347e1543-5865-4117-b72a-0c90aa4228af None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:07.913836 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347e1543-5865-4117-b72a-0c90aa4228af None admin] PATH_INFO: `/v3/users` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:07.914548 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347e1543-5865-4117-b72a-0c90aa4228af None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:07.916675 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347e1543-5865-4117-b72a-0c90aa4228af None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:07.935358 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 103/197] 10.222.0.46 () {60 vars in 1132 bytes} [Thu Jun 23 15:47:07 2022] POST /identity/v3/users => generated 297 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 15:47:07.935358 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347e1543-5865-4117-b72a-0c90aa4228af None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:07.944642 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0ac8ca-d835-4f97-9b62-69ca20b81b9d None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:07.980736 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:07.988366 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfc048f-cfcf-4618-8f5d-ff48a2a9244f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:07.989289 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfc048f-cfcf-4618-8f5d-ff48a2a9244f None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:07.989644 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfc048f-cfcf-4618-8f5d-ff48a2a9244f None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:07.990859 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfc048f-cfcf-4618-8f5d-ff48a2a9244f None admin] PATH_INFO: `/v3/users/233da9d863e245f0b59eef6794b16abd` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:07.991567 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfc048f-cfcf-4618-8f5d-ff48a2a9244f None admin] RBAC: Authorizing `identity:get_user(user_id=233da9d863e245f0b59eef6794b16abd)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:07.993086 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfc048f-cfcf-4618-8f5d-ff48a2a9244f None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:07.999930 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 95/198] 10.222.0.46 () {56 vars in 1177 bytes} [Thu Jun 23 15:47:07 2022] GET /identity/v3/users/233da9d863e245f0b59eef6794b16abd => generated 289 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:47:07.999930 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfc048f-cfcf-4618-8f5d-ff48a2a9244f None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:08.028465 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:08.079744 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347e1543-5865-4117-b72a-0c90aa4228af None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:08.123182 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b06eacd-81df-460f-97f7-7c587cfd8735 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:08.123851 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b06eacd-81df-460f-97f7-7c587cfd8735 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:08.124138 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b06eacd-81df-460f-97f7-7c587cfd8735 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:08.124415 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b06eacd-81df-460f-97f7-7c587cfd8735 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:08.131248 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfc048f-cfcf-4618-8f5d-ff48a2a9244f None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:08.175885 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c55310-c4e4-44af-acfb-966e81f9faa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:08.175885 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c55310-c4e4-44af-acfb-966e81f9faa2 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:08.175885 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c55310-c4e4-44af-acfb-966e81f9faa2 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:08.175885 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c55310-c4e4-44af-acfb-966e81f9faa2 None admin] PATH_INFO: `/v3/users/f084629de0a54e1493d2a025da535a8d` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:08.176785 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c55310-c4e4-44af-acfb-966e81f9faa2 None admin] RBAC: Authorizing `identity:get_user(user_id=f084629de0a54e1493d2a025da535a8d)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:08.178254 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c55310-c4e4-44af-acfb-966e81f9faa2 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:08.181270 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 96/199] 10.222.0.46 () {56 vars in 1177 bytes} [Thu Jun 23 15:47:07 2022] GET /identity/v3/users/f084629de0a54e1493d2a025da535a8d => generated 283 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:47:08.181270 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c55310-c4e4-44af-acfb-966e81f9faa2 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:08.210350 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b06eacd-81df-460f-97f7-7c587cfd8735 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:08.212769 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b06eacd-81df-460f-97f7-7c587cfd8735 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:08.272837 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:08.316796 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c55310-c4e4-44af-acfb-966e81f9faa2 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:08.318607 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:08.359093 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fa2ad9-c905-48df-8d22-e4067ef2a871 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:08.359921 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fa2ad9-c905-48df-8d22-e4067ef2a871 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:08.360246 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fa2ad9-c905-48df-8d22-e4067ef2a871 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:08.360541 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fa2ad9-c905-48df-8d22-e4067ef2a871 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:08.380016 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 104/200] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Jun 23 15:47:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 444 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 15:47:08.380907 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b06eacd-81df-460f-97f7-7c587cfd8735 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:08.441549 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fa2ad9-c905-48df-8d22-e4067ef2a871 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:08.443258 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fa2ad9-c905-48df-8d22-e4067ef2a871 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:08.501104 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b06eacd-81df-460f-97f7-7c587cfd8735 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:08.514319 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:08.543928 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54c4435-d35d-45b0-ba78-6815aa89f7a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:08.544973 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54c4435-d35d-45b0-ba78-6815aa89f7a2 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:08.545462 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54c4435-d35d-45b0-ba78-6815aa89f7a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:08.545992 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54c4435-d35d-45b0-ba78-6815aa89f7a2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:08.557119 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 97/201] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Jun 23 15:47:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 15:47:08.558332 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fa2ad9-c905-48df-8d22-e4067ef2a871 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:08.583677 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:08.628006 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54c4435-d35d-45b0-ba78-6815aa89f7a2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:08.630003 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54c4435-d35d-45b0-ba78-6815aa89f7a2 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:08.676922 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fa2ad9-c905-48df-8d22-e4067ef2a871 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:08.718403 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372180da-a3f6-45d6-9781-abf9c58f97f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:08.719673 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372180da-a3f6-45d6-9781-abf9c58f97f2 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:08.720105 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372180da-a3f6-45d6-9781-abf9c58f97f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:08.720544 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372180da-a3f6-45d6-9781-abf9c58f97f2 None admin] PATH_INFO: `/v3/projects/2240bbdc36ed4e59a7454b287b5891ab` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:08.721926 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372180da-a3f6-45d6-9781-abf9c58f97f2 None admin] RBAC: Authorizing `identity:get_project(project_id=2240bbdc36ed4e59a7454b287b5891ab)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:08.723887 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372180da-a3f6-45d6-9781-abf9c58f97f2 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:08.726220 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 98/202] 10.222.0.46 () {56 vars in 1186 bytes} [Thu Jun 23 15:47:08 2022] GET /identity/v3/projects/2240bbdc36ed4e59a7454b287b5891ab => generated 302 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:47:08.727898 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372180da-a3f6-45d6-9781-abf9c58f97f2 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:08.745386 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 105/203] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Jun 23 15:47:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 15:47:08.746066 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54c4435-d35d-45b0-ba78-6815aa89f7a2 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:08.767897 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:08.817818 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:08.845265 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372180da-a3f6-45d6-9781-abf9c58f97f2 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:08.863328 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54c4435-d35d-45b0-ba78-6815aa89f7a2 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:08.884328 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db60bd75-605c-41ac-9e5a-d85cf10c6136 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:08.885539 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db60bd75-605c-41ac-9e5a-d85cf10c6136 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:08.886023 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db60bd75-605c-41ac-9e5a-d85cf10c6136 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:08.886440 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db60bd75-605c-41ac-9e5a-d85cf10c6136 None admin] PATH_INFO: `/v3/projects/27f259ec3bd04186a70598926b09aa56` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:08.887972 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db60bd75-605c-41ac-9e5a-d85cf10c6136 None admin] RBAC: Authorizing `identity:get_project(project_id=27f259ec3bd04186a70598926b09aa56)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:08.889582 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db60bd75-605c-41ac-9e5a-d85cf10c6136 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:08.891973 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 99/204] 10.222.0.46 () {56 vars in 1186 bytes} [Thu Jun 23 15:47:08 2022] GET /identity/v3/projects/27f259ec3bd04186a70598926b09aa56 => generated 298 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:47:08.892935 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db60bd75-605c-41ac-9e5a-d85cf10c6136 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:08.903143 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffba4bf-aa98-433c-bafe-240dcbbd7919 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:08.904123 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffba4bf-aa98-433c-bafe-240dcbbd7919 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:08.904507 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffba4bf-aa98-433c-bafe-240dcbbd7919 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:08.904892 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffba4bf-aa98-433c-bafe-240dcbbd7919 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:08.905890 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffba4bf-aa98-433c-bafe-240dcbbd7919 None admin] RBAC: Adding query filter params (role.id=812d3c567c52469cb2b35709fbdc857c, scope.project.id=2240bbdc36ed4e59a7454b287b5891ab, user.id=f084629de0a54e1493d2a025da535a8d) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:47:08.906376 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffba4bf-aa98-433c-bafe-240dcbbd7919 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:08.908300 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffba4bf-aa98-433c-bafe-240dcbbd7919 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:08.914516 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 106/205] 10.222.0.46 () {56 vars in 1374 bytes} [Thu Jun 23 15:47:08 2022] GET /identity/v3/role_assignments?user.id=f084629de0a54e1493d2a025da535a8d&scope.project.id=2240bbdc36ed4e59a7454b287b5891ab&role.id=812d3c567c52469cb2b35709fbdc857c => generated 259 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:47:08.915617 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffba4bf-aa98-433c-bafe-240dcbbd7919 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:09.012957 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db60bd75-605c-41ac-9e5a-d85cf10c6136 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:09.016363 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:09.054782 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abff0cfb-4502-490c-88e8-d11a30eb2673 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:09.056028 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abff0cfb-4502-490c-88e8-d11a30eb2673 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:09.056614 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abff0cfb-4502-490c-88e8-d11a30eb2673 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:09.057098 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abff0cfb-4502-490c-88e8-d11a30eb2673 None admin] PATH_INFO: `/v3/projects/2240bbdc36ed4e59a7454b287b5891ab` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:09.058634 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abff0cfb-4502-490c-88e8-d11a30eb2673 None admin] RBAC: Authorizing `identity:get_project(project_id=2240bbdc36ed4e59a7454b287b5891ab)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:09.060468 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abff0cfb-4502-490c-88e8-d11a30eb2673 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:09.062816 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 100/206] 10.222.0.46 () {56 vars in 1186 bytes} [Thu Jun 23 15:47:08 2022] GET /identity/v3/projects/2240bbdc36ed4e59a7454b287b5891ab => generated 302 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:47:09.073940 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abff0cfb-4502-490c-88e8-d11a30eb2673 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:09.146995 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffba4bf-aa98-433c-bafe-240dcbbd7919 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:09.184191 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:09.195421 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b47ba75-38c7-4e54-ad93-6d0f0609f4e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:09.196633 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b47ba75-38c7-4e54-ad93-6d0f0609f4e8 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:09.196922 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b47ba75-38c7-4e54-ad93-6d0f0609f4e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:09.197211 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b47ba75-38c7-4e54-ad93-6d0f0609f4e8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:09.197919 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abff0cfb-4502-490c-88e8-d11a30eb2673 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:09.198113 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b47ba75-38c7-4e54-ad93-6d0f0609f4e8 None admin] RBAC: Adding query filter params (role.id=38bbc1cebab84cac9b213ffba44bb4a6, scope.project.id=27f259ec3bd04186a70598926b09aa56, user.id=233da9d863e245f0b59eef6794b16abd) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:47:09.198373 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b47ba75-38c7-4e54-ad93-6d0f0609f4e8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:09.199912 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b47ba75-38c7-4e54-ad93-6d0f0609f4e8 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:09.207455 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 107/207] 10.222.0.46 () {56 vars in 1374 bytes} [Thu Jun 23 15:47:08 2022] GET /identity/v3/role_assignments?user.id=233da9d863e245f0b59eef6794b16abd&scope.project.id=27f259ec3bd04186a70598926b09aa56&role.id=38bbc1cebab84cac9b213ffba44bb4a6 => generated 608 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:47:09.259412 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bca4411-ddcd-416b-b879-d120f4f7973f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:09.260257 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bca4411-ddcd-416b-b879-d120f4f7973f None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:09.260575 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bca4411-ddcd-416b-b879-d120f4f7973f None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:09.260859 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bca4411-ddcd-416b-b879-d120f4f7973f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:09.261750 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bca4411-ddcd-416b-b879-d120f4f7973f None admin] RBAC: Adding query filter params (role.id=d4d557a3ec4e434f8e83499c5fcc7136, scope.project.id=2240bbdc36ed4e59a7454b287b5891ab, user.id=f084629de0a54e1493d2a025da535a8d) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:47:09.262195 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bca4411-ddcd-416b-b879-d120f4f7973f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:09.264016 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bca4411-ddcd-416b-b879-d120f4f7973f None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:09.271569 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 101/208] 10.222.0.46 () {56 vars in 1374 bytes} [Thu Jun 23 15:47:09 2022] GET /identity/v3/role_assignments?user.id=f084629de0a54e1493d2a025da535a8d&scope.project.id=2240bbdc36ed4e59a7454b287b5891ab&role.id=d4d557a3ec4e434f8e83499c5fcc7136 => generated 259 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:47:10.272631 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e972097c-f38f-4084-868d-55f3d2e7a369 None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:10.273427 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e972097c-f38f-4084-868d-55f3d2e7a369 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:10.273427 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e972097c-f38f-4084-868d-55f3d2e7a369 None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:10.275457 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 108/209] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jun 23 15:47:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 15:47:10.281067 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79533359-db5b-4a3a-9062-d38412fd4eca None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:10.281585 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79533359-db5b-4a3a-9062-d38412fd4eca None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:10.281677 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79533359-db5b-4a3a-9062-d38412fd4eca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:10.300247 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-79533359-db5b-4a3a-9062-d38412fd4eca None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:47:10.301468 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-79533359-db5b-4a3a-9062-d38412fd4eca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:47:10.317806 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e723168-438e-42f9-9220-ae1f04f97383 None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:10.318098 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e723168-438e-42f9-9220-ae1f04f97383 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:10.318387 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e723168-438e-42f9-9220-ae1f04f97383 None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:10.319837 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 109/210] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jun 23 15:47:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 15:47:10.325476 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f63ff84-4512-4d75-bec8-fb77f4ce2849 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:10.325778 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f63ff84-4512-4d75-bec8-fb77f4ce2849 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:10.326063 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f63ff84-4512-4d75-bec8-fb77f4ce2849 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:10.329911 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:10.345401 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-4f63ff84-4512-4d75-bec8-fb77f4ce2849 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:47:10.346772 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-4f63ff84-4512-4d75-bec8-fb77f4ce2849 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:47:10.382579 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 102/211] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jun 23 15:47:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 15:47:10.407255 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79533359-db5b-4a3a-9062-d38412fd4eca None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:10.430781 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 110/212] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jun 23 15:47:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 15:47:10.453144 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f63ff84-4512-4d75-bec8-fb77f4ce2849 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:10.528027 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79533359-db5b-4a3a-9062-d38412fd4eca None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:10.574958 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf30e5dc-41ba-4b73-a2a9-e0406c07dc96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:10.575868 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf30e5dc-41ba-4b73-a2a9-e0406c07dc96 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:10.576164 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf30e5dc-41ba-4b73-a2a9-e0406c07dc96 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:10.576784 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf30e5dc-41ba-4b73-a2a9-e0406c07dc96 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:10.578547 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f63ff84-4512-4d75-bec8-fb77f4ce2849 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:10.588903 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf30e5dc-41ba-4b73-a2a9-e0406c07dc96 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 15:47:10.596795 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-bf30e5dc-41ba-4b73-a2a9-e0406c07dc96 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 23 15:47:10.596795 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:47:10.596795 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:47:10.596795 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:47:10.596795 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:47:10.596795 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:47:10.596795 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:47:10.596795 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:47:10.596795 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:47:10.596795 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:47:10.596795 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:47:10.596795 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:47:10.596795 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 15:47:10.596795 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 15:47:10.596795 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 15:47:10.596795 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 15:47:10.597673 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 15:47:10.597673 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 15:47:10.597673 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 15:47:10.597673 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 15:47:10.597673 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:47:10.597673 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:47:10.597673 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:47:10.597673 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:47:10.597673 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:47:10.597673 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:47:10.597673 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:47:10.597673 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Jun 23 15:47:10.597673 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:47:10.597673 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:47:10.597673 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:47:10.597673 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:47:10.598481 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:47:10.598481 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:47:10.598481 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:47:10.598481 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:47:10.598481 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 15:47:10.598481 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 15:47:10.598481 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 15:47:10.598481 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 15:47:10.598481 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 15:47:10.598481 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 15:47:10.598481 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 23 15:47:10.598481 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Jun 23 15:47:10.602856 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 103/213] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Jun 23 15:47:10 2022] GET /identity/v3/roles/member => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:47:10.604129 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf30e5dc-41ba-4b73-a2a9-e0406c07dc96 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:10.605406 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:10.623541 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a1714c-2d8e-410b-a0e9-a7dfaa2290e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:10.624788 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a1714c-2d8e-410b-a0e9-a7dfaa2290e1 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:10.625516 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a1714c-2d8e-410b-a0e9-a7dfaa2290e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:10.625942 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a1714c-2d8e-410b-a0e9-a7dfaa2290e1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:10.627330 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a1714c-2d8e-410b-a0e9-a7dfaa2290e1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:10.629682 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a1714c-2d8e-410b-a0e9-a7dfaa2290e1 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:10.631665 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:10.631870 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 111/214] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Jun 23 15:47:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 15:47:10.639046 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a1714c-2d8e-410b-a0e9-a7dfaa2290e1 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:10.745055 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf30e5dc-41ba-4b73-a2a9-e0406c07dc96 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:10.761386 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a1714c-2d8e-410b-a0e9-a7dfaa2290e1 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:10.792966 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527db856-4ec3-4a3a-b5f8-97641bccbe7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:10.794494 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527db856-4ec3-4a3a-b5f8-97641bccbe7a None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:10.794964 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527db856-4ec3-4a3a-b5f8-97641bccbe7a None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:10.795667 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527db856-4ec3-4a3a-b5f8-97641bccbe7a None admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:10.796690 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527db856-4ec3-4a3a-b5f8-97641bccbe7a None admin] RBAC: Adding query filter params (name=member) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:47:10.797168 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527db856-4ec3-4a3a-b5f8-97641bccbe7a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:10.799222 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527db856-4ec3-4a3a-b5f8-97641bccbe7a None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:10.799656 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37309da0-8c5d-4615-b9f6-c57e730c196f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:10.800449 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37309da0-8c5d-4615-b9f6-c57e730c196f None admin] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:10.800830 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37309da0-8c5d-4615-b9f6-c57e730c196f None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:10.801201 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37309da0-8c5d-4615-b9f6-c57e730c196f None admin] PATH_INFO: `/v3/users` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:10.802011 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37309da0-8c5d-4615-b9f6-c57e730c196f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:10.803601 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37309da0-8c5d-4615-b9f6-c57e730c196f None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:10.809994 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 104/215] 10.222.0.46 () {56 vars in 1101 bytes} [Thu Jun 23 15:47:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:47:10.816660 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527db856-4ec3-4a3a-b5f8-97641bccbe7a None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:10.840353 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 112/216] 10.222.0.46 () {60 vars in 1132 bytes} [Thu Jun 23 15:47:10 2022] POST /identity/v3/users => generated 297 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 15:47:10.923360 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:10.942821 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527db856-4ec3-4a3a-b5f8-97641bccbe7a None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:10.982297 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e647de74-d93a-49b1-9c01-57e3f90dd49c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:10.983645 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e647de74-d93a-49b1-9c01-57e3f90dd49c None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:10.984149 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e647de74-d93a-49b1-9c01-57e3f90dd49c None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:10.984733 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e647de74-d93a-49b1-9c01-57e3f90dd49c None admin] PATH_INFO: `/v3/users/8acd9db80c3b4c03af741ef235678038` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:11.003770 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e647de74-d93a-49b1-9c01-57e3f90dd49c None admin] RBAC: Authorizing `identity:get_user(user_id=8acd9db80c3b4c03af741ef235678038)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:11.006523 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e647de74-d93a-49b1-9c01-57e3f90dd49c None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:11.010397 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 105/217] 10.222.0.46 () {56 vars in 1177 bytes} [Thu Jun 23 15:47:10 2022] GET /identity/v3/users/8acd9db80c3b4c03af741ef235678038 => generated 297 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 15:47:11.016443 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37309da0-8c5d-4615-b9f6-c57e730c196f None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:11.100451 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:11.195301 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37309da0-8c5d-4615-b9f6-c57e730c196f None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:11.224192 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbff6773-9e15-45bb-bd2c-c068e1771c55 None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:11.224931 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbff6773-9e15-45bb-bd2c-c068e1771c55 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:11.225427 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbff6773-9e15-45bb-bd2c-c068e1771c55 None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:11.228227 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 106/218] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jun 23 15:47:11 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 15:47:11.233902 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60661054-c930-472f-9f3a-683cfca243ed None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:11.234262 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60661054-c930-472f-9f3a-683cfca243ed None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:11.234618 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60661054-c930-472f-9f3a-683cfca243ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:11.235771 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a99e2d-d2fc-4070-9891-86593251099a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:11.236277 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a99e2d-d2fc-4070-9891-86593251099a None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:11.236562 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a99e2d-d2fc-4070-9891-86593251099a None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:11.236839 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a99e2d-d2fc-4070-9891-86593251099a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:11.258342 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-60661054-c930-472f-9f3a-683cfca243ed None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:47:11.259622 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-60661054-c930-472f-9f3a-683cfca243ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:47:11.317857 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a99e2d-d2fc-4070-9891-86593251099a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:11.319516 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a99e2d-d2fc-4070-9891-86593251099a None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:11.334957 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 107/219] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jun 23 15:47:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 15:47:11.360416 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60661054-c930-472f-9f3a-683cfca243ed None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:11.374721 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:11.455139 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:11.458488 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 113/220] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Jun 23 15:47:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 442 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 15:47:11.467233 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a99e2d-d2fc-4070-9891-86593251099a None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:11.501528 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60661054-c930-472f-9f3a-683cfca243ed None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:11.580015 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33130be-fa27-463c-9bc5-63315314586b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:11.582726 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33130be-fa27-463c-9bc5-63315314586b None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:11.582726 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33130be-fa27-463c-9bc5-63315314586b None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:11.582726 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33130be-fa27-463c-9bc5-63315314586b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:11.602929 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33130be-fa27-463c-9bc5-63315314586b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 15:47:11.619198 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-d33130be-fa27-463c-9bc5-63315314586b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 15:47:11.619198 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:47:11.619198 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:47:11.619198 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:47:11.619198 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:47:11.619198 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:47:11.619198 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:47:11.619198 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:47:11.619198 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:47:11.619198 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:47:11.619198 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:47:11.619198 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:47:11.619198 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 15:47:11.619198 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 15:47:11.619198 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 15:47:11.619198 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 15:47:11.620109 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 15:47:11.620109 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 15:47:11.620109 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 15:47:11.620109 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 15:47:11.620109 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:47:11.620109 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:47:11.620109 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:47:11.620109 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:47:11.620109 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:47:11.620109 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:47:11.620109 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:47:11.620109 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Jun 23 15:47:11.620109 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:47:11.620109 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:47:11.620109 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:47:11.620109 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:47:11.620895 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:47:11.620895 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:47:11.620895 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:47:11.620895 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:47:11.620895 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 15:47:11.620895 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 15:47:11.620895 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 15:47:11.620895 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 15:47:11.620895 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 15:47:11.620895 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 15:47:11.620895 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 15:47:11.620895 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Jun 23 15:47:11.620895 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 108/221] 10.222.0.46 () {56 vars in 1096 bytes} [Thu Jun 23 15:47:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:47:11.626902 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7262bf-83f3-43cd-bfbe-19ab8aa60d8e None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:11.627196 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7262bf-83f3-43cd-bfbe-19ab8aa60d8e None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:11.627481 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7262bf-83f3-43cd-bfbe-19ab8aa60d8e None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:11.628866 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 109/222] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jun 23 15:47:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 15:47:11.633098 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7262bf-83f3-43cd-bfbe-19ab8aa60d8e None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:11.660097 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a99e2d-d2fc-4070-9891-86593251099a None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:11.710821 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24856eac-df9c-4ce4-b758-6dc6fd8806ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:11.712379 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24856eac-df9c-4ce4-b758-6dc6fd8806ef None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:11.713130 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24856eac-df9c-4ce4-b758-6dc6fd8806ef None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:11.714047 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24856eac-df9c-4ce4-b758-6dc6fd8806ef None admin] PATH_INFO: `/v3/projects/2240bbdc36ed4e59a7454b287b5891ab` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:11.716377 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24856eac-df9c-4ce4-b758-6dc6fd8806ef None admin] RBAC: Authorizing `identity:get_project(project_id=2240bbdc36ed4e59a7454b287b5891ab)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:11.719311 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24856eac-df9c-4ce4-b758-6dc6fd8806ef None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:11.722327 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 114/223] 10.222.0.46 () {56 vars in 1186 bytes} [Thu Jun 23 15:47:11 2022] GET /identity/v3/projects/2240bbdc36ed4e59a7454b287b5891ab => generated 302 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:47:11.723861 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447a099f-3996-4355-9fc5-db4260ef79dc None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:11.724386 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447a099f-3996-4355-9fc5-db4260ef79dc None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:11.724859 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447a099f-3996-4355-9fc5-db4260ef79dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:11.748569 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-447a099f-3996-4355-9fc5-db4260ef79dc None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:47:11.750728 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-447a099f-3996-4355-9fc5-db4260ef79dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:47:11.763642 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:11.765540 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7262bf-83f3-43cd-bfbe-19ab8aa60d8e None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:11.826908 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7645cd-4dd7-4079-abb9-34d25de02107 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:11.827841 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7645cd-4dd7-4079-abb9-34d25de02107 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:11.828269 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7645cd-4dd7-4079-abb9-34d25de02107 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:11.828688 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7645cd-4dd7-4079-abb9-34d25de02107 None admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:11.830052 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7645cd-4dd7-4079-abb9-34d25de02107 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:47:11.830785 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7645cd-4dd7-4079-abb9-34d25de02107 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:11.832498 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7645cd-4dd7-4079-abb9-34d25de02107 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:11.834725 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:11.844286 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 115/224] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jun 23 15:47:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 15:47:11.845623 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 110/225] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Jun 23 15:47:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:47:11.847600 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447a099f-3996-4355-9fc5-db4260ef79dc None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:11.854734 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7645cd-4dd7-4079-abb9-34d25de02107 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:11.988592 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7645cd-4dd7-4079-abb9-34d25de02107 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:12.008060 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447a099f-3996-4355-9fc5-db4260ef79dc None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:12.028026 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c1e164-50d9-47d1-992a-db4c26bf7259 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:12.029110 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c1e164-50d9-47d1-992a-db4c26bf7259 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:12.029491 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c1e164-50d9-47d1-992a-db4c26bf7259 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:12.029864 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c1e164-50d9-47d1-992a-db4c26bf7259 None admin] PATH_INFO: `/v3/users/f084629de0a54e1493d2a025da535a8d` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:12.031205 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c1e164-50d9-47d1-992a-db4c26bf7259 None admin] RBAC: Authorizing `identity:get_user(user_id=f084629de0a54e1493d2a025da535a8d)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:12.032766 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c1e164-50d9-47d1-992a-db4c26bf7259 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:12.039101 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 111/226] 10.222.0.46 () {56 vars in 1177 bytes} [Thu Jun 23 15:47:11 2022] GET /identity/v3/users/f084629de0a54e1493d2a025da535a8d => generated 283 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 15:47:12.039101 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c1e164-50d9-47d1-992a-db4c26bf7259 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:12.054819 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5657a135-6cdf-4487-a3e6-5ac853e3674c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:12.055524 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5657a135-6cdf-4487-a3e6-5ac853e3674c None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:12.055803 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5657a135-6cdf-4487-a3e6-5ac853e3674c None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:12.056075 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5657a135-6cdf-4487-a3e6-5ac853e3674c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:12.057108 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5657a135-6cdf-4487-a3e6-5ac853e3674c None admin] RBAC: Adding query filter params (role.id=e2a5bbf3200a4d4091d72bc7ff78c34d, scope.project.id=2240bbdc36ed4e59a7454b287b5891ab, user.id=8acd9db80c3b4c03af741ef235678038) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:47:12.057550 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5657a135-6cdf-4487-a3e6-5ac853e3674c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:12.059601 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5657a135-6cdf-4487-a3e6-5ac853e3674c None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:12.065521 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 116/227] 10.222.0.46 () {56 vars in 1374 bytes} [Thu Jun 23 15:47:11 2022] GET /identity/v3/role_assignments?user.id=8acd9db80c3b4c03af741ef235678038&scope.project.id=2240bbdc36ed4e59a7454b287b5891ab&role.id=e2a5bbf3200a4d4091d72bc7ff78c34d => generated 259 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:47:12.072091 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5657a135-6cdf-4487-a3e6-5ac853e3674c None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:12.092500 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:12.129830 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:12.180125 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c1e164-50d9-47d1-992a-db4c26bf7259 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:12.224834 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3191378-a354-4d99-abf9-6ffc561a226b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:12.226392 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3191378-a354-4d99-abf9-6ffc561a226b None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:12.226953 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3191378-a354-4d99-abf9-6ffc561a226b None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:12.227451 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3191378-a354-4d99-abf9-6ffc561a226b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:12.243234 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3191378-a354-4d99-abf9-6ffc561a226b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 15:47:12.254108 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-b3191378-a354-4d99-abf9-6ffc561a226b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 15:47:12.254108 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:47:12.254108 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:47:12.254108 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:47:12.254108 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:47:12.254108 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:47:12.254108 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:47:12.254108 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:47:12.254108 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:47:12.254108 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:47:12.254108 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:47:12.254108 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:47:12.254108 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 15:47:12.254108 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 15:47:12.254108 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 15:47:12.254108 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 15:47:12.255116 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 15:47:12.255116 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 15:47:12.255116 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 15:47:12.255116 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 15:47:12.255116 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:47:12.255116 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:47:12.255116 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:47:12.255116 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:47:12.255116 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:47:12.255116 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:47:12.255116 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:47:12.255116 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Jun 23 15:47:12.255116 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:47:12.255116 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:47:12.255116 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:47:12.255116 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:47:12.255930 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:47:12.255930 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:47:12.255930 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:47:12.255930 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:47:12.255930 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 15:47:12.255930 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 15:47:12.255930 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 15:47:12.255930 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 15:47:12.255930 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 15:47:12.255930 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 15:47:12.255930 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 15:47:12.255930 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Jun 23 15:47:12.259640 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 112/228] 10.222.0.46 () {56 vars in 1114 bytes} [Thu Jun 23 15:47:12 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:47:12.268800 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3191378-a354-4d99-abf9-6ffc561a226b None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:12.304486 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5657a135-6cdf-4487-a3e6-5ac853e3674c None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:12.371499 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960f67b2-98c0-455a-85f9-40dc8c022628 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:12.373272 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960f67b2-98c0-455a-85f9-40dc8c022628 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:12.373765 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960f67b2-98c0-455a-85f9-40dc8c022628 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:12.374250 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960f67b2-98c0-455a-85f9-40dc8c022628 None admin] PATH_INFO: `/v3/projects/2240bbdc36ed4e59a7454b287b5891ab` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:12.375984 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960f67b2-98c0-455a-85f9-40dc8c022628 None admin] RBAC: Authorizing `identity:get_project(project_id=2240bbdc36ed4e59a7454b287b5891ab)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:12.377680 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960f67b2-98c0-455a-85f9-40dc8c022628 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:12.379904 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 117/229] 10.222.0.46 () {56 vars in 1186 bytes} [Thu Jun 23 15:47:12 2022] GET /identity/v3/projects/2240bbdc36ed4e59a7454b287b5891ab => generated 302 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:47:12.386474 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960f67b2-98c0-455a-85f9-40dc8c022628 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:12.425048 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3191378-a354-4d99-abf9-6ffc561a226b None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:12.467358 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:12.478383 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:12.507521 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960f67b2-98c0-455a-85f9-40dc8c022628 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:12.519400 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c03785-f14b-4db6-81aa-cde50c9b4722 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:12.520319 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c03785-f14b-4db6-81aa-cde50c9b4722 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:12.520740 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c03785-f14b-4db6-81aa-cde50c9b4722 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:12.521160 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c03785-f14b-4db6-81aa-cde50c9b4722 None admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:12.522065 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c03785-f14b-4db6-81aa-cde50c9b4722 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:47:12.522541 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c03785-f14b-4db6-81aa-cde50c9b4722 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:12.524148 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c03785-f14b-4db6-81aa-cde50c9b4722 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:12.545913 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8dbe405-5b3b-4c70-9817-929e9ef1a199 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:12.547202 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dbe405-5b3b-4c70-9817-929e9ef1a199 None admin] REQUEST_METHOD: `PUT` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:12.547786 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dbe405-5b3b-4c70-9817-929e9ef1a199 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:12.548279 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dbe405-5b3b-4c70-9817-929e9ef1a199 None admin] PATH_INFO: `/v3/projects/2240bbdc36ed4e59a7454b287b5891ab/users/f084629de0a54e1493d2a025da535a8d/roles/812d3c567c52469cb2b35709fbdc857c` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:12.549301 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 113/230] 10.222.0.46 () {56 vars in 1111 bytes} [Thu Jun 23 15:47:12 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:47:12.550818 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8dbe405-5b3b-4c70-9817-929e9ef1a199 None admin] RBAC: Authorizing `identity:create_grant(project_id=2240bbdc36ed4e59a7454b287b5891ab, user_id=f084629de0a54e1493d2a025da535a8d, role_id=812d3c567c52469cb2b35709fbdc857c)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:12.552486 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8dbe405-5b3b-4c70-9817-929e9ef1a199 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:12.554340 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c03785-f14b-4db6-81aa-cde50c9b4722 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:12.580789 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 118/231] 10.222.0.46 () {58 vars in 1439 bytes} [Thu Jun 23 15:47:12 2022] PUT /identity/v3/projects/2240bbdc36ed4e59a7454b287b5891ab/users/f084629de0a54e1493d2a025da535a8d/roles/812d3c567c52469cb2b35709fbdc857c => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 15:47:12.689844 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c03785-f14b-4db6-81aa-cde50c9b4722 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:12.731491 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7baa01a6-13c7-49c5-9d39-df20f09d05b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:12.732418 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baa01a6-13c7-49c5-9d39-df20f09d05b3 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:12.732707 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baa01a6-13c7-49c5-9d39-df20f09d05b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:12.732986 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baa01a6-13c7-49c5-9d39-df20f09d05b3 None admin] PATH_INFO: `/v3/users/f084629de0a54e1493d2a025da535a8d` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:12.734308 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7baa01a6-13c7-49c5-9d39-df20f09d05b3 None admin] RBAC: Authorizing `identity:get_user(user_id=f084629de0a54e1493d2a025da535a8d)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:12.735830 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7baa01a6-13c7-49c5-9d39-df20f09d05b3 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:12.737735 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 114/232] 10.222.0.46 () {56 vars in 1177 bytes} [Thu Jun 23 15:47:12 2022] GET /identity/v3/users/f084629de0a54e1493d2a025da535a8d => generated 283 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:47:12.744381 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8dbe405-5b3b-4c70-9817-929e9ef1a199 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:12.880682 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8dbe405-5b3b-4c70-9817-929e9ef1a199 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:12.917644 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e0d74d-b7e3-41ad-afc3-397020e41b09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:12.918524 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e0d74d-b7e3-41ad-afc3-397020e41b09 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:12.918848 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e0d74d-b7e3-41ad-afc3-397020e41b09 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:12.919131 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e0d74d-b7e3-41ad-afc3-397020e41b09 None admin] PATH_INFO: `/v3/projects/2240bbdc36ed4e59a7454b287b5891ab` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:12.920367 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e0d74d-b7e3-41ad-afc3-397020e41b09 None admin] RBAC: Authorizing `identity:get_project(project_id=2240bbdc36ed4e59a7454b287b5891ab)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:12.921947 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e0d74d-b7e3-41ad-afc3-397020e41b09 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:12.924052 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 119/233] 10.222.0.46 () {56 vars in 1186 bytes} [Thu Jun 23 15:47:12 2022] GET /identity/v3/projects/2240bbdc36ed4e59a7454b287b5891ab => generated 302 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:47:12.932083 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7baa01a6-13c7-49c5-9d39-df20f09d05b3 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:13.021163 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:13.081893 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7baa01a6-13c7-49c5-9d39-df20f09d05b3 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:13.130502 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5acf48-dac4-4315-b250-93446b60e28b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:13.131530 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5acf48-dac4-4315-b250-93446b60e28b None admin] REQUEST_METHOD: `PUT` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:13.131754 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5acf48-dac4-4315-b250-93446b60e28b None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:13.132032 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5acf48-dac4-4315-b250-93446b60e28b None admin] PATH_INFO: `/v3/projects/2240bbdc36ed4e59a7454b287b5891ab/users/f084629de0a54e1493d2a025da535a8d/roles/d4d557a3ec4e434f8e83499c5fcc7136` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:13.134351 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5acf48-dac4-4315-b250-93446b60e28b None admin] RBAC: Authorizing `identity:create_grant(project_id=2240bbdc36ed4e59a7454b287b5891ab, user_id=f084629de0a54e1493d2a025da535a8d, role_id=d4d557a3ec4e434f8e83499c5fcc7136)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:13.135899 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5acf48-dac4-4315-b250-93446b60e28b None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:13.159486 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 115/234] 10.222.0.46 () {58 vars in 1439 bytes} [Thu Jun 23 15:47:12 2022] PUT /identity/v3/projects/2240bbdc36ed4e59a7454b287b5891ab/users/f084629de0a54e1493d2a025da535a8d/roles/d4d557a3ec4e434f8e83499c5fcc7136 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 15:47:13.583382 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981f8e69-af91-4135-a68e-904afcb56c59 None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:13.583817 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981f8e69-af91-4135-a68e-904afcb56c59 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:13.584009 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981f8e69-af91-4135-a68e-904afcb56c59 None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:13.586007 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 120/235] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jun 23 15:47:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 15:47:13.596362 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c2082e-5715-4b39-b795-fcc545f933d0 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:13.596876 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61078ed2-9ca6-40b8-bf32-9cd6d67a8d7e None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:13.597167 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61078ed2-9ca6-40b8-bf32-9cd6d67a8d7e None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:13.597448 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61078ed2-9ca6-40b8-bf32-9cd6d67a8d7e None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:13.597783 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c2082e-5715-4b39-b795-fcc545f933d0 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:13.598248 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c2082e-5715-4b39-b795-fcc545f933d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:13.599126 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 121/236] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jun 23 15:47:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 15:47:13.606210 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec596da-8000-4039-9999-4a8b3617e785 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:13.606210 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec596da-8000-4039-9999-4a8b3617e785 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:13.606210 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec596da-8000-4039-9999-4a8b3617e785 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:13.610915 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:13.624567 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-fec596da-8000-4039-9999-4a8b3617e785 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:47:13.626488 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-fec596da-8000-4039-9999-4a8b3617e785 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:47:13.631777 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-23c2082e-5715-4b39-b795-fcc545f933d0 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:47:13.632978 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-23c2082e-5715-4b39-b795-fcc545f933d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:47:13.708662 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 122/237] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jun 23 15:47:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 15:47:13.745644 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 116/238] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jun 23 15:47:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 15:47:13.752305 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec596da-8000-4039-9999-4a8b3617e785 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:13.769821 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c2082e-5715-4b39-b795-fcc545f933d0 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:13.844040 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:13.922154 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec596da-8000-4039-9999-4a8b3617e785 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:13.944300 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c2082e-5715-4b39-b795-fcc545f933d0 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:13.988275 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:14.002472 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78cbdea-00fc-415c-9908-733bada44a2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:14.003735 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78cbdea-00fc-415c-9908-733bada44a2b None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:14.004222 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78cbdea-00fc-415c-9908-733bada44a2b None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:14.004696 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78cbdea-00fc-415c-9908-733bada44a2b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:14.015241 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae1eba1-cd9c-428d-be7c-9b57a538a9fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:14.017774 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae1eba1-cd9c-428d-be7c-9b57a538a9fa None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:14.018901 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae1eba1-cd9c-428d-be7c-9b57a538a9fa None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:14.019572 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae1eba1-cd9c-428d-be7c-9b57a538a9fa None admin] PATH_INFO: `/v3/domains/default` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:14.022461 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae1eba1-cd9c-428d-be7c-9b57a538a9fa None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:14.024286 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae1eba1-cd9c-428d-be7c-9b57a538a9fa None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:14.030827 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 117/239] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Jun 23 15:47:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 15:47:14.031633 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78cbdea-00fc-415c-9908-733bada44a2b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 15:47:14.035395 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae1eba1-cd9c-428d-be7c-9b57a538a9fa None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:14.043823 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-a78cbdea-00fc-415c-9908-733bada44a2b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 23 15:47:14.043823 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:47:14.043823 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:47:14.043823 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:47:14.043823 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:47:14.043823 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:47:14.043823 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:47:14.043823 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:47:14.043823 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:47:14.043823 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:47:14.043823 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:47:14.043823 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:47:14.043823 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 15:47:14.043823 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 15:47:14.043823 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 15:47:14.043823 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 15:47:14.044704 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 15:47:14.044704 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 15:47:14.044704 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 15:47:14.044704 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 15:47:14.044704 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:47:14.044704 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:47:14.044704 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:47:14.044704 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:47:14.044704 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:47:14.044704 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:47:14.044704 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:47:14.044704 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Jun 23 15:47:14.044704 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:47:14.044704 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:47:14.044704 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:47:14.044704 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:47:14.045529 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:47:14.045529 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:47:14.045529 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:47:14.045529 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:47:14.045529 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 15:47:14.045529 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 15:47:14.045529 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 15:47:14.045529 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 15:47:14.045529 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 15:47:14.045529 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 15:47:14.045529 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 23 15:47:14.045529 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Jun 23 15:47:14.056103 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 123/240] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Jun 23 15:47:13 2022] GET /identity/v3/roles/reader => generated 84 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:47:14.063900 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78cbdea-00fc-415c-9908-733bada44a2b None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:14.153435 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae1eba1-cd9c-428d-be7c-9b57a538a9fa None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:14.172324 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:14.192670 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026aad23-8960-460d-a48c-096af5b3e34a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:14.193817 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026aad23-8960-460d-a48c-096af5b3e34a None admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:14.194291 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026aad23-8960-460d-a48c-096af5b3e34a None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:14.194650 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026aad23-8960-460d-a48c-096af5b3e34a None admin] PATH_INFO: `/v3/users` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:14.195449 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026aad23-8960-460d-a48c-096af5b3e34a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:14.197277 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026aad23-8960-460d-a48c-096af5b3e34a None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:14.225718 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 118/241] 10.222.0.46 () {60 vars in 1132 bytes} [Thu Jun 23 15:47:14 2022] POST /identity/v3/users => generated 293 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 15:47:14.269975 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78cbdea-00fc-415c-9908-733bada44a2b None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:14.318882 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d357eb9d-6c6f-4db0-830d-638fc4b6f58d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:14.319592 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d357eb9d-6c6f-4db0-830d-638fc4b6f58d None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:14.319889 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d357eb9d-6c6f-4db0-830d-638fc4b6f58d None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:14.320154 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d357eb9d-6c6f-4db0-830d-638fc4b6f58d None admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:14.320971 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d357eb9d-6c6f-4db0-830d-638fc4b6f58d None admin] RBAC: Adding query filter params (name=reader) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:47:14.321282 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d357eb9d-6c6f-4db0-830d-638fc4b6f58d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:14.323644 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d357eb9d-6c6f-4db0-830d-638fc4b6f58d None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:14.333419 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 124/242] 10.222.0.46 () {56 vars in 1101 bytes} [Thu Jun 23 15:47:14 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:47:14.344001 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026aad23-8960-460d-a48c-096af5b3e34a None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:14.470213 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026aad23-8960-460d-a48c-096af5b3e34a None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:14.511146 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d71257f-a07c-402f-b42d-0d78f77e1a39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:14.512392 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d71257f-a07c-402f-b42d-0d78f77e1a39 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:14.512870 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d71257f-a07c-402f-b42d-0d78f77e1a39 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:14.513369 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d71257f-a07c-402f-b42d-0d78f77e1a39 None admin] PATH_INFO: `/v3/users/648dab2f743c4481befe6a87f21263d3` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:14.529835 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d71257f-a07c-402f-b42d-0d78f77e1a39 None admin] RBAC: Authorizing `identity:get_user(user_id=648dab2f743c4481befe6a87f21263d3)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:14.531718 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d71257f-a07c-402f-b42d-0d78f77e1a39 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:14.533997 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 119/243] 10.222.0.46 () {56 vars in 1177 bytes} [Thu Jun 23 15:47:14 2022] GET /identity/v3/users/648dab2f743c4481befe6a87f21263d3 => generated 297 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:47:14.541301 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d357eb9d-6c6f-4db0-830d-638fc4b6f58d None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:14.622463 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:14.648391 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ee68de-2f2e-47d7-9985-f721ccd89514 None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:14.649898 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ee68de-2f2e-47d7-9985-f721ccd89514 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:14.650440 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ee68de-2f2e-47d7-9985-f721ccd89514 None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:14.652772 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 120/244] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jun 23 15:47:14 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 15:47:14.660314 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20632ff-d1a2-4097-8598-ce58e3d9af16 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:14.660607 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20632ff-d1a2-4097-8598-ce58e3d9af16 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:14.660894 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20632ff-d1a2-4097-8598-ce58e3d9af16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:14.672726 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d357eb9d-6c6f-4db0-830d-638fc4b6f58d None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:14.700217 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-d20632ff-d1a2-4097-8598-ce58e3d9af16 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:47:14.701682 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-d20632ff-d1a2-4097-8598-ce58e3d9af16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:47:14.718735 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4041126f-1891-4af6-96da-be2ab3314e12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:14.719410 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4041126f-1891-4af6-96da-be2ab3314e12 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:14.719693 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4041126f-1891-4af6-96da-be2ab3314e12 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:14.719972 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4041126f-1891-4af6-96da-be2ab3314e12 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:14.739194 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:14.812157 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4041126f-1891-4af6-96da-be2ab3314e12 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:14.813890 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4041126f-1891-4af6-96da-be2ab3314e12 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:14.827224 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 121/245] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jun 23 15:47:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 15:47:14.860038 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20632ff-d1a2-4097-8598-ce58e3d9af16 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:14.915917 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:14.948864 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 125/246] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Jun 23 15:47:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 15:47:14.956760 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4041126f-1891-4af6-96da-be2ab3314e12 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:14.989443 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20632ff-d1a2-4097-8598-ce58e3d9af16 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:15.069805 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fea323a-a2d4-4969-8556-c314f9440634 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:15.070671 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fea323a-a2d4-4969-8556-c314f9440634 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:15.070984 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fea323a-a2d4-4969-8556-c314f9440634 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:15.071262 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fea323a-a2d4-4969-8556-c314f9440634 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:15.084715 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4041126f-1891-4af6-96da-be2ab3314e12 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:15.111815 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fea323a-a2d4-4969-8556-c314f9440634 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 15:47:15.121004 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-5fea323a-a2d4-4969-8556-c314f9440634 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 23 15:47:15.121004 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:47:15.121004 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:47:15.121004 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:47:15.121004 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:47:15.121004 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:47:15.121004 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:47:15.121004 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:47:15.121004 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:47:15.121004 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:47:15.121004 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:47:15.121004 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:47:15.121004 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 15:47:15.121004 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 15:47:15.121004 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 15:47:15.121004 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 15:47:15.122202 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 15:47:15.122202 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 15:47:15.122202 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 15:47:15.122202 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 15:47:15.122202 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:47:15.122202 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:47:15.122202 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:47:15.122202 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:47:15.122202 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:47:15.122202 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:47:15.122202 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:47:15.122202 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Jun 23 15:47:15.122202 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:47:15.122202 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:47:15.122202 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:47:15.122202 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:47:15.123206 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:47:15.123206 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:47:15.123206 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:47:15.123206 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:47:15.123206 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 15:47:15.123206 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 15:47:15.123206 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 15:47:15.123206 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 15:47:15.123206 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 15:47:15.123206 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 15:47:15.123206 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 23 15:47:15.123206 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Jun 23 15:47:15.123206 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 122/247] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Jun 23 15:47:14 2022] GET /identity/v3/roles/member => generated 84 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 15:47:15.128923 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fea323a-a2d4-4969-8556-c314f9440634 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:15.130380 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0709bb31-e43d-4dba-b796-a09cd81c734c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:15.131322 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0709bb31-e43d-4dba-b796-a09cd81c734c None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:15.131616 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0709bb31-e43d-4dba-b796-a09cd81c734c None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:15.131902 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0709bb31-e43d-4dba-b796-a09cd81c734c None admin] PATH_INFO: `/v3/projects/2240bbdc36ed4e59a7454b287b5891ab` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:15.133613 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0709bb31-e43d-4dba-b796-a09cd81c734c None admin] RBAC: Authorizing `identity:get_project(project_id=2240bbdc36ed4e59a7454b287b5891ab)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:15.136802 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0709bb31-e43d-4dba-b796-a09cd81c734c None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:15.143487 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 126/248] 10.222.0.46 () {56 vars in 1186 bytes} [Thu Jun 23 15:47:14 2022] GET /identity/v3/projects/2240bbdc36ed4e59a7454b287b5891ab => generated 302 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 15:47:15.149547 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0709bb31-e43d-4dba-b796-a09cd81c734c None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:15.184579 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:15.188469 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:15.333751 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0709bb31-e43d-4dba-b796-a09cd81c734c None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:15.419882 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fea323a-a2d4-4969-8556-c314f9440634 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:15.433161 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace1d271-a133-432b-aa01-1632c9a4052b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:15.433844 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace1d271-a133-432b-aa01-1632c9a4052b None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:15.434123 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace1d271-a133-432b-aa01-1632c9a4052b None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:15.434399 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace1d271-a133-432b-aa01-1632c9a4052b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:15.435277 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace1d271-a133-432b-aa01-1632c9a4052b None admin] RBAC: Adding query filter params (role.id=38bbc1cebab84cac9b213ffba44bb4a6, scope.project.id=2240bbdc36ed4e59a7454b287b5891ab, user.id=648dab2f743c4481befe6a87f21263d3) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:47:15.435586 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace1d271-a133-432b-aa01-1632c9a4052b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:15.437078 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace1d271-a133-432b-aa01-1632c9a4052b None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:15.458845 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 127/249] 10.222.0.46 () {56 vars in 1374 bytes} [Thu Jun 23 15:47:15 2022] GET /identity/v3/role_assignments?user.id=648dab2f743c4481befe6a87f21263d3&scope.project.id=2240bbdc36ed4e59a7454b287b5891ab&role.id=38bbc1cebab84cac9b213ffba44bb4a6 => generated 259 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 15:47:15.460590 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da795418-c31c-4f4f-aedc-b2a77b62b758 None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:15.460925 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da795418-c31c-4f4f-aedc-b2a77b62b758 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:15.461297 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da795418-c31c-4f4f-aedc-b2a77b62b758 None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:15.462975 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 128/250] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jun 23 15:47:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 15:47:15.472321 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7c395b-555e-4e63-8eb0-4fd5851087b8 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:15.472648 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7c395b-555e-4e63-8eb0-4fd5851087b8 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:15.472927 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7c395b-555e-4e63-8eb0-4fd5851087b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:15.512139 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-3c7c395b-555e-4e63-8eb0-4fd5851087b8 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:47:15.513564 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-3c7c395b-555e-4e63-8eb0-4fd5851087b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:47:15.542580 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e14443e-671e-4093-9c74-1d040ad86431 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:15.544463 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e14443e-671e-4093-9c74-1d040ad86431 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:15.545682 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e14443e-671e-4093-9c74-1d040ad86431 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:15.547253 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e14443e-671e-4093-9c74-1d040ad86431 None admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:15.549350 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e14443e-671e-4093-9c74-1d040ad86431 None admin] RBAC: Adding query filter params (name=member) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:47:15.553425 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e14443e-671e-4093-9c74-1d040ad86431 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:15.555357 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e14443e-671e-4093-9c74-1d040ad86431 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:15.592958 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 123/251] 10.222.0.46 () {56 vars in 1101 bytes} [Thu Jun 23 15:47:15 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 465 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:47:15.599829 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e14443e-671e-4093-9c74-1d040ad86431 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:15.697433 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 129/252] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jun 23 15:47:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 15:47:15.746419 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7c395b-555e-4e63-8eb0-4fd5851087b8 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:15.779097 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:15.782929 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:15.826443 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e14443e-671e-4093-9c74-1d040ad86431 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:15.871415 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7c395b-555e-4e63-8eb0-4fd5851087b8 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:15.907683 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c07fc2c-6ec4-41cb-bbb6-687924a6a3ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:15.908255 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc69f96-43b7-4844-b4db-efdb65e607a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:15.908487 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c07fc2c-6ec4-41cb-bbb6-687924a6a3ab None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:15.908781 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c07fc2c-6ec4-41cb-bbb6-687924a6a3ab None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:15.909060 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c07fc2c-6ec4-41cb-bbb6-687924a6a3ab None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:15.909158 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc69f96-43b7-4844-b4db-efdb65e607a9 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:15.909410 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc69f96-43b7-4844-b4db-efdb65e607a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:15.909683 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc69f96-43b7-4844-b4db-efdb65e607a9 None admin] PATH_INFO: `/v3/users/8acd9db80c3b4c03af741ef235678038` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:15.915635 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc69f96-43b7-4844-b4db-efdb65e607a9 None admin] RBAC: Authorizing `identity:get_user(user_id=8acd9db80c3b4c03af741ef235678038)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:15.917206 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc69f96-43b7-4844-b4db-efdb65e607a9 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:15.919192 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 124/253] 10.222.0.46 () {56 vars in 1177 bytes} [Thu Jun 23 15:47:15 2022] GET /identity/v3/users/8acd9db80c3b4c03af741ef235678038 => generated 297 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:47:15.921149 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c07fc2c-6ec4-41cb-bbb6-687924a6a3ab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 15:47:15.924150 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc69f96-43b7-4844-b4db-efdb65e607a9 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:15.929013 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-1c07fc2c-6ec4-41cb-bbb6-687924a6a3ab None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 15:47:15.929013 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:47:15.929013 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:47:15.929013 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:47:15.929013 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:47:15.929013 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:47:15.929013 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:47:15.929013 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:47:15.929013 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:47:15.929013 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:47:15.929013 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:47:15.929013 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:47:15.929013 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 15:47:15.929013 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 15:47:15.929013 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 15:47:15.929013 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 15:47:15.929935 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 15:47:15.929935 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 15:47:15.929935 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 15:47:15.929935 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 15:47:15.929935 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:47:15.929935 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:47:15.929935 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:47:15.929935 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:47:15.929935 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:47:15.929935 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:47:15.929935 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:47:15.929935 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Jun 23 15:47:15.929935 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:47:15.929935 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:47:15.929935 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:47:15.929935 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:47:15.931736 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:47:15.931736 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:47:15.931736 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:47:15.931736 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:47:15.931736 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 15:47:15.931736 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 15:47:15.931736 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 15:47:15.931736 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 15:47:15.931736 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 15:47:15.931736 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 15:47:15.931736 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 15:47:15.931736 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Jun 23 15:47:15.931736 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 130/254] 10.222.0.46 () {56 vars in 1096 bytes} [Thu Jun 23 15:47:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 15:47:15.938710 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c07fc2c-6ec4-41cb-bbb6-687924a6a3ab None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:16.023315 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:16.048998 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c07fc2c-6ec4-41cb-bbb6-687924a6a3ab None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:16.084980 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c84c91-928f-4ebb-9586-63c8bffe4fb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:16.085658 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c84c91-928f-4ebb-9586-63c8bffe4fb9 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:16.085944 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c84c91-928f-4ebb-9586-63c8bffe4fb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:16.094769 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c84c91-928f-4ebb-9586-63c8bffe4fb9 None admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:16.094769 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c84c91-928f-4ebb-9586-63c8bffe4fb9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:47:16.094769 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c84c91-928f-4ebb-9586-63c8bffe4fb9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:16.094769 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c84c91-928f-4ebb-9586-63c8bffe4fb9 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:16.099225 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 131/255] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Jun 23 15:47:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:47:16.103517 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc69f96-43b7-4844-b4db-efdb65e607a9 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:16.107855 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ce9957-1346-4ae8-b69d-33ba673638a4 None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:16.107855 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ce9957-1346-4ae8-b69d-33ba673638a4 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:16.107855 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ce9957-1346-4ae8-b69d-33ba673638a4 None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:16.107855 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 132/256] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jun 23 15:47:16 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 15:47:16.108436 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ce9957-1346-4ae8-b69d-33ba673638a4 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:16.169737 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1718d3e1-e914-4bb0-9b7a-b7b85de37a3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:16.171009 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1718d3e1-e914-4bb0-9b7a-b7b85de37a3e None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:16.171495 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1718d3e1-e914-4bb0-9b7a-b7b85de37a3e None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:16.171963 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1718d3e1-e914-4bb0-9b7a-b7b85de37a3e None admin] PATH_INFO: `/v3/projects/2240bbdc36ed4e59a7454b287b5891ab` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:16.176508 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1718d3e1-e914-4bb0-9b7a-b7b85de37a3e None admin] RBAC: Authorizing `identity:get_project(project_id=2240bbdc36ed4e59a7454b287b5891ab)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:16.178008 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1718d3e1-e914-4bb0-9b7a-b7b85de37a3e None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:16.180079 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 125/257] 10.222.0.46 () {56 vars in 1186 bytes} [Thu Jun 23 15:47:15 2022] GET /identity/v3/projects/2240bbdc36ed4e59a7454b287b5891ab => generated 302 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 15:47:16.188271 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3fa2ec-2ed6-44ba-a6c9-b29a3c4a3276 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:16.188550 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3fa2ec-2ed6-44ba-a6c9-b29a3c4a3276 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:16.188835 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3fa2ec-2ed6-44ba-a6c9-b29a3c4a3276 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:16.223753 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-2f3fa2ec-2ed6-44ba-a6c9-b29a3c4a3276 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:47:16.224976 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-2f3fa2ec-2ed6-44ba-a6c9-b29a3c4a3276 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:47:16.257002 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:16.259404 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ce9957-1346-4ae8-b69d-33ba673638a4 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:16.297561 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c21d6f5-7db3-4b88-b62a-cfaaab3723d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:16.298566 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c21d6f5-7db3-4b88-b62a-cfaaab3723d7 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:16.298893 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c21d6f5-7db3-4b88-b62a-cfaaab3723d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:16.299177 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c21d6f5-7db3-4b88-b62a-cfaaab3723d7 None admin] PATH_INFO: `/v3/users/f084629de0a54e1493d2a025da535a8d` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:16.300418 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c21d6f5-7db3-4b88-b62a-cfaaab3723d7 None admin] RBAC: Authorizing `identity:get_user(user_id=f084629de0a54e1493d2a025da535a8d)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:16.301949 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c21d6f5-7db3-4b88-b62a-cfaaab3723d7 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:16.303980 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 133/258] 10.222.0.46 () {56 vars in 1177 bytes} [Thu Jun 23 15:47:16 2022] GET /identity/v3/users/f084629de0a54e1493d2a025da535a8d => generated 283 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:47:16.306236 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c21d6f5-7db3-4b88-b62a-cfaaab3723d7 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:16.331104 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:16.425532 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 126/259] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jun 23 15:47:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 15:47:16.426809 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3fa2ec-2ed6-44ba-a6c9-b29a3c4a3276 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:16.446565 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c21d6f5-7db3-4b88-b62a-cfaaab3723d7 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:16.526400 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c72526-b4ec-4410-9d73-fe45fc0a1906 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:16.527702 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c72526-b4ec-4410-9d73-fe45fc0a1906 None admin] REQUEST_METHOD: `PUT` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:16.528177 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c72526-b4ec-4410-9d73-fe45fc0a1906 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:16.528647 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c72526-b4ec-4410-9d73-fe45fc0a1906 None admin] PATH_INFO: `/v3/projects/2240bbdc36ed4e59a7454b287b5891ab/users/8acd9db80c3b4c03af741ef235678038/roles/e2a5bbf3200a4d4091d72bc7ff78c34d` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:16.533406 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c72526-b4ec-4410-9d73-fe45fc0a1906 None admin] RBAC: Authorizing `identity:create_grant(project_id=2240bbdc36ed4e59a7454b287b5891ab, user_id=8acd9db80c3b4c03af741ef235678038, role_id=e2a5bbf3200a4d4091d72bc7ff78c34d)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:16.535281 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c72526-b4ec-4410-9d73-fe45fc0a1906 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:16.553486 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 134/260] 10.222.0.46 () {58 vars in 1439 bytes} [Thu Jun 23 15:47:16 2022] PUT /identity/v3/projects/2240bbdc36ed4e59a7454b287b5891ab/users/8acd9db80c3b4c03af741ef235678038/roles/e2a5bbf3200a4d4091d72bc7ff78c34d => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 15:47:16.558016 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c72526-b4ec-4410-9d73-fe45fc0a1906 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:16.624292 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3fa2ec-2ed6-44ba-a6c9-b29a3c4a3276 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:16.653065 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:16.697059 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c72526-b4ec-4410-9d73-fe45fc0a1906 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:16.707590 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1229e8b-cfa2-4365-a33f-95ec0ed94f84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:16.708266 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1229e8b-cfa2-4365-a33f-95ec0ed94f84 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:16.708551 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1229e8b-cfa2-4365-a33f-95ec0ed94f84 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:16.708829 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1229e8b-cfa2-4365-a33f-95ec0ed94f84 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:16.743035 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e60b16-0c67-4bc7-8b26-86726c70331a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:16.743732 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e60b16-0c67-4bc7-8b26-86726c70331a None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:16.743981 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e60b16-0c67-4bc7-8b26-86726c70331a None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:16.744359 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e60b16-0c67-4bc7-8b26-86726c70331a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:16.744762 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:16.787451 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e60b16-0c67-4bc7-8b26-86726c70331a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 15:47:16.806125 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-27e60b16-0c67-4bc7-8b26-86726c70331a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 15:47:16.806125 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:47:16.806125 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:47:16.806125 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:47:16.806125 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:47:16.806125 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:47:16.806125 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:47:16.806125 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:47:16.806125 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:47:16.806125 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:47:16.806125 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:47:16.806125 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:47:16.806125 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 15:47:16.806125 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 15:47:16.806125 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 15:47:16.806125 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 15:47:16.807105 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 15:47:16.807105 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 15:47:16.807105 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 15:47:16.807105 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 15:47:16.807105 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:47:16.807105 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:47:16.807105 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:47:16.807105 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:47:16.807105 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:47:16.807105 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:47:16.807105 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:47:16.807105 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Jun 23 15:47:16.807105 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:47:16.807105 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:47:16.807105 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:47:16.807105 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:47:16.807902 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:47:16.807902 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:47:16.807902 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:47:16.807902 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:47:16.807902 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 15:47:16.807902 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 15:47:16.807902 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 15:47:16.807902 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 15:47:16.807902 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 15:47:16.807902 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 15:47:16.807902 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 15:47:16.807902 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Jun 23 15:47:16.809123 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 135/261] 10.222.0.46 () {56 vars in 1114 bytes} [Thu Jun 23 15:47:16 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:47:16.815332 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e60b16-0c67-4bc7-8b26-86726c70331a None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:16.875792 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1229e8b-cfa2-4365-a33f-95ec0ed94f84 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:16.877293 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1229e8b-cfa2-4365-a33f-95ec0ed94f84 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:16.948761 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e60b16-0c67-4bc7-8b26-86726c70331a None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:16.983092 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:16.991272 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98d2ba6-f2c0-46a1-b33b-cc16ce5e9dc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:16.992022 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98d2ba6-f2c0-46a1-b33b-cc16ce5e9dc2 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:16.992343 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98d2ba6-f2c0-46a1-b33b-cc16ce5e9dc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:16.992628 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98d2ba6-f2c0-46a1-b33b-cc16ce5e9dc2 None admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:16.993423 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98d2ba6-f2c0-46a1-b33b-cc16ce5e9dc2 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:47:16.993738 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98d2ba6-f2c0-46a1-b33b-cc16ce5e9dc2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:16.995205 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98d2ba6-f2c0-46a1-b33b-cc16ce5e9dc2 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:17.004920 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 136/262] 10.222.0.46 () {56 vars in 1111 bytes} [Thu Jun 23 15:47:16 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:47:17.011033 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98d2ba6-f2c0-46a1-b33b-cc16ce5e9dc2 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:17.117904 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 127/263] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Jun 23 15:47:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 692 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 15:47:17.123955 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1229e8b-cfa2-4365-a33f-95ec0ed94f84 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:17.233405 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98d2ba6-f2c0-46a1-b33b-cc16ce5e9dc2 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:17.245339 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:17.311853 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf52b615-7e66-4981-aff3-8bf745ff32b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:17.313100 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf52b615-7e66-4981-aff3-8bf745ff32b6 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:17.313597 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf52b615-7e66-4981-aff3-8bf745ff32b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:17.314071 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf52b615-7e66-4981-aff3-8bf745ff32b6 None admin] PATH_INFO: `/v3/users/f084629de0a54e1493d2a025da535a8d` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:17.315562 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf52b615-7e66-4981-aff3-8bf745ff32b6 None admin] RBAC: Authorizing `identity:get_user(user_id=f084629de0a54e1493d2a025da535a8d)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:17.317229 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf52b615-7e66-4981-aff3-8bf745ff32b6 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:17.319136 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 137/264] 10.222.0.46 () {56 vars in 1177 bytes} [Thu Jun 23 15:47:17 2022] GET /identity/v3/users/f084629de0a54e1493d2a025da535a8d => generated 283 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:47:17.326158 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf52b615-7e66-4981-aff3-8bf745ff32b6 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:17.379965 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1229e8b-cfa2-4365-a33f-95ec0ed94f84 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:17.436176 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:17.463424 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bfe9bb-16ac-450f-9b98-d3ef2a3a8f10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:17.464322 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bfe9bb-16ac-450f-9b98-d3ef2a3a8f10 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:17.464616 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bfe9bb-16ac-450f-9b98-d3ef2a3a8f10 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:17.464884 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bfe9bb-16ac-450f-9b98-d3ef2a3a8f10 None admin] PATH_INFO: `/v3/projects/2240bbdc36ed4e59a7454b287b5891ab` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:17.467315 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bfe9bb-16ac-450f-9b98-d3ef2a3a8f10 None admin] RBAC: Authorizing `identity:get_project(project_id=2240bbdc36ed4e59a7454b287b5891ab)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:17.468875 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bfe9bb-16ac-450f-9b98-d3ef2a3a8f10 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:17.470968 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 128/265] 10.222.0.46 () {56 vars in 1186 bytes} [Thu Jun 23 15:47:17 2022] GET /identity/v3/projects/2240bbdc36ed4e59a7454b287b5891ab => generated 302 bytes in 347 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 15:47:17.484575 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bfe9bb-16ac-450f-9b98-d3ef2a3a8f10 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:17.540707 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf52b615-7e66-4981-aff3-8bf745ff32b6 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:17.612895 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40a726c-7f96-4a5a-9801-c3995a82194b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:17.613563 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40a726c-7f96-4a5a-9801-c3995a82194b None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:17.613849 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40a726c-7f96-4a5a-9801-c3995a82194b None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:17.614127 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40a726c-7f96-4a5a-9801-c3995a82194b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:17.701092 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40a726c-7f96-4a5a-9801-c3995a82194b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:17.702582 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40a726c-7f96-4a5a-9801-c3995a82194b None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:17.760769 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:17.764438 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bfe9bb-16ac-450f-9b98-d3ef2a3a8f10 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:17.793545 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:17.808793 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 138/266] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Jun 23 15:47:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 483 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 23 15:47:17.810102 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216a6134-9c7c-4448-b8a6-53ebb8705ce2 None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:17.810393 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216a6134-9c7c-4448-b8a6-53ebb8705ce2 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:17.810670 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216a6134-9c7c-4448-b8a6-53ebb8705ce2 None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:17.811968 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 139/267] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jun 23 15:47:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 15:47:17.818421 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b56c6d6-99cf-4a04-af51-91cdc8764138 None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:17.818421 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b56c6d6-99cf-4a04-af51-91cdc8764138 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:17.818421 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b56c6d6-99cf-4a04-af51-91cdc8764138 None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:17.818421 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 140/268] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jun 23 15:47:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 15:47:17.818421 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b56c6d6-99cf-4a04-af51-91cdc8764138 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:17.829730 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed9bb2f-eb06-46ff-a798-ad249d2c16bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:17.830861 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed9bb2f-eb06-46ff-a798-ad249d2c16bb None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:17.831358 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed9bb2f-eb06-46ff-a798-ad249d2c16bb None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:17.831848 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed9bb2f-eb06-46ff-a798-ad249d2c16bb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:17.833141 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed9bb2f-eb06-46ff-a798-ad249d2c16bb None admin] RBAC: Adding query filter params (role.id=812d3c567c52469cb2b35709fbdc857c, scope.project.id=2240bbdc36ed4e59a7454b287b5891ab, user.id=f084629de0a54e1493d2a025da535a8d) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:47:17.833664 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed9bb2f-eb06-46ff-a798-ad249d2c16bb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:17.835421 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed9bb2f-eb06-46ff-a798-ad249d2c16bb None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:17.842648 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 129/269] 10.222.0.46 () {56 vars in 1374 bytes} [Thu Jun 23 15:47:17 2022] GET /identity/v3/role_assignments?user.id=f084629de0a54e1493d2a025da535a8d&scope.project.id=2240bbdc36ed4e59a7454b287b5891ab&role.id=812d3c567c52469cb2b35709fbdc857c => generated 608 bytes in 359 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 15:47:17.844345 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2791829-7090-4d77-806b-81415ef94153 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:17.844845 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2791829-7090-4d77-806b-81415ef94153 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:17.845362 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2791829-7090-4d77-806b-81415ef94153 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:17.864046 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-f2791829-7090-4d77-806b-81415ef94153 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:47:17.865804 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-f2791829-7090-4d77-806b-81415ef94153 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:47:17.932603 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b56c6d6-99cf-4a04-af51-91cdc8764138 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:17.941848 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 130/270] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jun 23 15:47:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 15:47:17.948282 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11600e03-f711-4406-a66f-4688d6d608d7 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:17.948884 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11600e03-f711-4406-a66f-4688d6d608d7 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:17.949370 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11600e03-f711-4406-a66f-4688d6d608d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:17.977424 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-11600e03-f711-4406-a66f-4688d6d608d7 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:47:17.979056 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-11600e03-f711-4406-a66f-4688d6d608d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:47:17.980253 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6071e3-1950-44bc-828d-7737a1ba7218 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:17.981185 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6071e3-1950-44bc-828d-7737a1ba7218 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:17.981465 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6071e3-1950-44bc-828d-7737a1ba7218 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:17.981737 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6071e3-1950-44bc-828d-7737a1ba7218 None admin] PATH_INFO: `/v3/projects/2240bbdc36ed4e59a7454b287b5891ab` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:17.983105 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6071e3-1950-44bc-828d-7737a1ba7218 None admin] RBAC: Authorizing `identity:get_project(project_id=2240bbdc36ed4e59a7454b287b5891ab)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:17.984768 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6071e3-1950-44bc-828d-7737a1ba7218 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:17.986530 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 141/271] 10.222.0.46 () {56 vars in 1186 bytes} [Thu Jun 23 15:47:17 2022] GET /identity/v3/projects/2240bbdc36ed4e59a7454b287b5891ab => generated 302 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:47:17.991778 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6071e3-1950-44bc-828d-7737a1ba7218 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:18.049761 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:18.097057 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 131/272] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jun 23 15:47:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 15:47:18.098459 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11600e03-f711-4406-a66f-4688d6d608d7 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:18.138965 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:18.187537 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6071e3-1950-44bc-828d-7737a1ba7218 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:18.215723 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11600e03-f711-4406-a66f-4688d6d608d7 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:18.230250 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1e67b4-eeff-4f89-a378-0a1a94b7bc62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:18.231097 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1e67b4-eeff-4f89-a378-0a1a94b7bc62 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:18.231378 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1e67b4-eeff-4f89-a378-0a1a94b7bc62 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:18.231650 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1e67b4-eeff-4f89-a378-0a1a94b7bc62 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:18.244222 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1e67b4-eeff-4f89-a378-0a1a94b7bc62 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 15:47:18.252462 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-db1e67b4-eeff-4f89-a378-0a1a94b7bc62 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 23 15:47:18.252462 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:47:18.252462 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:47:18.252462 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:47:18.252462 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:47:18.252462 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:47:18.252462 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:47:18.252462 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:47:18.252462 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:47:18.252462 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:47:18.252462 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:47:18.252462 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:47:18.252462 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 15:47:18.252462 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 15:47:18.252462 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 15:47:18.252462 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 15:47:18.253404 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 15:47:18.253404 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 15:47:18.253404 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 15:47:18.253404 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 15:47:18.253404 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:47:18.253404 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:47:18.253404 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:47:18.253404 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:47:18.253404 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:47:18.253404 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:47:18.253404 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:47:18.253404 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Jun 23 15:47:18.253404 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:47:18.253404 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:47:18.253404 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:47:18.253404 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:47:18.254173 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:47:18.254173 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:47:18.254173 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:47:18.254173 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:47:18.254173 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 15:47:18.254173 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 15:47:18.254173 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 15:47:18.254173 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 15:47:18.254173 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 15:47:18.254173 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 15:47:18.254173 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 23 15:47:18.254173 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Jun 23 15:47:18.258078 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 142/273] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Jun 23 15:47:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 15:47:18.259038 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1e67b4-eeff-4f89-a378-0a1a94b7bc62 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:18.263101 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c462fa-13df-4e28-ba64-049171d8ab9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:18.264046 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c462fa-13df-4e28-ba64-049171d8ab9e None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:18.264541 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c462fa-13df-4e28-ba64-049171d8ab9e None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:18.265023 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c462fa-13df-4e28-ba64-049171d8ab9e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:18.266109 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c462fa-13df-4e28-ba64-049171d8ab9e None admin] RBAC: Adding query filter params (role.id=d4d557a3ec4e434f8e83499c5fcc7136, scope.project.id=2240bbdc36ed4e59a7454b287b5891ab, user.id=f084629de0a54e1493d2a025da535a8d) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:47:18.266651 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c462fa-13df-4e28-ba64-049171d8ab9e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:18.268898 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c462fa-13df-4e28-ba64-049171d8ab9e None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:18.278364 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 132/274] 10.222.0.46 () {56 vars in 1374 bytes} [Thu Jun 23 15:47:18 2022] GET /identity/v3/role_assignments?user.id=f084629de0a54e1493d2a025da535a8d&scope.project.id=2240bbdc36ed4e59a7454b287b5891ab&role.id=d4d557a3ec4e434f8e83499c5fcc7136 => generated 608 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:47:18.279792 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c462fa-13df-4e28-ba64-049171d8ab9e None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:18.363416 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:18.392174 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1e67b4-eeff-4f89-a378-0a1a94b7bc62 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:18.423397 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c462fa-13df-4e28-ba64-049171d8ab9e None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:18.425325 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:18.445496 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91539921-71c1-4ad6-9cdf-9d4e5d456531 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:18.446470 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91539921-71c1-4ad6-9cdf-9d4e5d456531 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:18.446789 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91539921-71c1-4ad6-9cdf-9d4e5d456531 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:18.447263 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91539921-71c1-4ad6-9cdf-9d4e5d456531 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:18.448518 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91539921-71c1-4ad6-9cdf-9d4e5d456531 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:18.450428 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91539921-71c1-4ad6-9cdf-9d4e5d456531 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:18.452747 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 143/275] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Jun 23 15:47:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:47:18.456075 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f56eaf-33f9-4f6b-8d6d-10d6bb813004 None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:18.456366 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f56eaf-33f9-4f6b-8d6d-10d6bb813004 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:18.456649 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f56eaf-33f9-4f6b-8d6d-10d6bb813004 None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:18.458356 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 144/276] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jun 23 15:47:18 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 15:47:18.459161 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f56eaf-33f9-4f6b-8d6d-10d6bb813004 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:18.481802 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f46792-9466-4e92-8bb1-d2902c079e02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:18.482508 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f46792-9466-4e92-8bb1-d2902c079e02 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:18.482827 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f46792-9466-4e92-8bb1-d2902c079e02 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:18.483108 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f46792-9466-4e92-8bb1-d2902c079e02 None admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:18.483882 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f46792-9466-4e92-8bb1-d2902c079e02 None admin] RBAC: Adding query filter params (name=member) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:47:18.484196 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f46792-9466-4e92-8bb1-d2902c079e02 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:18.485621 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f46792-9466-4e92-8bb1-d2902c079e02 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:18.501357 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 133/277] 10.222.0.46 () {56 vars in 1101 bytes} [Thu Jun 23 15:47:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:47:18.502721 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e525b932-659d-44f6-a9f7-461a68a621d9 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:18.503039 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e525b932-659d-44f6-a9f7-461a68a621d9 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:18.503330 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e525b932-659d-44f6-a9f7-461a68a621d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:18.526234 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-e525b932-659d-44f6-a9f7-461a68a621d9 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:47:18.527504 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-e525b932-659d-44f6-a9f7-461a68a621d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:47:18.601697 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f56eaf-33f9-4f6b-8d6d-10d6bb813004 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:18.619362 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 134/278] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jun 23 15:47:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 15:47:18.619362 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e525b932-659d-44f6-a9f7-461a68a621d9 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:18.653696 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6ad113-301c-4804-9b7c-75a455b507c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:18.654921 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6ad113-301c-4804-9b7c-75a455b507c2 None admin] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:18.655682 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6ad113-301c-4804-9b7c-75a455b507c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:18.656415 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6ad113-301c-4804-9b7c-75a455b507c2 None admin] PATH_INFO: `/v3/users` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:18.657331 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6ad113-301c-4804-9b7c-75a455b507c2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:18.659946 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6ad113-301c-4804-9b7c-75a455b507c2 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:18.684857 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 145/279] 10.222.0.46 () {60 vars in 1132 bytes} [Thu Jun 23 15:47:18 2022] POST /identity/v3/users => generated 293 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 15:47:18.686078 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6ad113-301c-4804-9b7c-75a455b507c2 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:18.708711 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:18.735770 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:18.748031 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e525b932-659d-44f6-a9f7-461a68a621d9 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:18.794682 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adae59db-55d3-4db0-8611-2d14567970a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:18.795786 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adae59db-55d3-4db0-8611-2d14567970a4 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:18.796185 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adae59db-55d3-4db0-8611-2d14567970a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:18.796869 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adae59db-55d3-4db0-8611-2d14567970a4 None admin] PATH_INFO: `/v3/users/47443d2110cd4bfb83a197c0664325a3` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:18.817566 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adae59db-55d3-4db0-8611-2d14567970a4 None admin] RBAC: Authorizing `identity:get_user(user_id=47443d2110cd4bfb83a197c0664325a3)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:18.820230 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adae59db-55d3-4db0-8611-2d14567970a4 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:18.827855 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 135/280] 10.222.0.46 () {56 vars in 1177 bytes} [Thu Jun 23 15:47:18 2022] GET /identity/v3/users/47443d2110cd4bfb83a197c0664325a3 => generated 293 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:47:18.829680 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adae59db-55d3-4db0-8611-2d14567970a4 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:18.852530 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6ad113-301c-4804-9b7c-75a455b507c2 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:18.899605 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb84493c-e8c1-4d0d-8df6-82a4db7c8385 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:18.900475 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb84493c-e8c1-4d0d-8df6-82a4db7c8385 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:18.900781 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb84493c-e8c1-4d0d-8df6-82a4db7c8385 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:18.901331 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb84493c-e8c1-4d0d-8df6-82a4db7c8385 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:18.913070 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb84493c-e8c1-4d0d-8df6-82a4db7c8385 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 15:47:18.921302 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-bb84493c-e8c1-4d0d-8df6-82a4db7c8385 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 23 15:47:18.921302 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:47:18.921302 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:47:18.921302 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:47:18.921302 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:47:18.921302 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:47:18.921302 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:47:18.921302 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:47:18.921302 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:47:18.921302 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:47:18.921302 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:47:18.921302 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:47:18.921302 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 15:47:18.921302 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 15:47:18.921302 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 15:47:18.921302 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 15:47:18.922182 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 15:47:18.922182 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 15:47:18.922182 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 15:47:18.922182 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 15:47:18.922182 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:47:18.922182 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:47:18.922182 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:47:18.922182 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:47:18.922182 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:47:18.922182 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:47:18.922182 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:47:18.922182 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Jun 23 15:47:18.922182 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:47:18.922182 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:47:18.922182 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:47:18.922182 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:47:18.923169 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:47:18.923169 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:47:18.923169 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:47:18.923169 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:47:18.923169 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 15:47:18.923169 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 15:47:18.923169 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 15:47:18.923169 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 15:47:18.923169 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 15:47:18.923169 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 15:47:18.923169 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 23 15:47:18.923169 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Jun 23 15:47:18.923169 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 146/281] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Jun 23 15:47:18 2022] GET /identity/v3/roles/reader => generated 84 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:47:18.928657 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb84493c-e8c1-4d0d-8df6-82a4db7c8385 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:18.969374 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adae59db-55d3-4db0-8611-2d14567970a4 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:19.017606 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:19.020039 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ad9de6-1eb4-4bfa-aee4-37e3fa5b65cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:19.020979 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ad9de6-1eb4-4bfa-aee4-37e3fa5b65cf None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:19.021261 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ad9de6-1eb4-4bfa-aee4-37e3fa5b65cf None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:19.021532 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ad9de6-1eb4-4bfa-aee4-37e3fa5b65cf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:19.022375 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ad9de6-1eb4-4bfa-aee4-37e3fa5b65cf None admin] RBAC: Adding query filter params (role.id=e2a5bbf3200a4d4091d72bc7ff78c34d, user.id=47443d2110cd4bfb83a197c0664325a3, scope.system=all) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:47:19.022682 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ad9de6-1eb4-4bfa-aee4-37e3fa5b65cf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:19.024203 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ad9de6-1eb4-4bfa-aee4-37e3fa5b65cf None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:19.031963 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 136/282] 10.222.0.46 () {56 vars in 1308 bytes} [Thu Jun 23 15:47:18 2022] GET /identity/v3/role_assignments?user.id=47443d2110cd4bfb83a197c0664325a3&scope.system=all&role.id=e2a5bbf3200a4d4091d72bc7ff78c34d => generated 226 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:47:19.061955 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb84493c-e8c1-4d0d-8df6-82a4db7c8385 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:19.107755 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d93cfaf-c009-41d4-980f-93bfbe55fc31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:19.108462 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d93cfaf-c009-41d4-980f-93bfbe55fc31 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:19.108749 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d93cfaf-c009-41d4-980f-93bfbe55fc31 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:19.109029 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d93cfaf-c009-41d4-980f-93bfbe55fc31 None admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:19.109825 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d93cfaf-c009-41d4-980f-93bfbe55fc31 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:47:19.110136 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d93cfaf-c009-41d4-980f-93bfbe55fc31 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:19.111706 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d93cfaf-c009-41d4-980f-93bfbe55fc31 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:19.122569 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 147/283] 10.222.0.46 () {56 vars in 1101 bytes} [Thu Jun 23 15:47:18 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:47:19.130290 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ad9de6-1eb4-4bfa-aee4-37e3fa5b65cf None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:19.160564 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:19.261410 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ad9de6-1eb4-4bfa-aee4-37e3fa5b65cf None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:19.298250 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de4ae45-63ac-4f28-9f2b-1aafc34178e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:19.299156 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de4ae45-63ac-4f28-9f2b-1aafc34178e9 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:19.299437 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de4ae45-63ac-4f28-9f2b-1aafc34178e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:19.299710 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de4ae45-63ac-4f28-9f2b-1aafc34178e9 None admin] PATH_INFO: `/v3/users/648dab2f743c4481befe6a87f21263d3` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:19.300944 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de4ae45-63ac-4f28-9f2b-1aafc34178e9 None admin] RBAC: Authorizing `identity:get_user(user_id=648dab2f743c4481befe6a87f21263d3)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:19.302385 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de4ae45-63ac-4f28-9f2b-1aafc34178e9 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:19.304420 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 137/284] 10.222.0.46 () {56 vars in 1177 bytes} [Thu Jun 23 15:47:19 2022] GET /identity/v3/users/648dab2f743c4481befe6a87f21263d3 => generated 297 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:47:19.311243 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d93cfaf-c009-41d4-980f-93bfbe55fc31 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:19.369089 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b78f78-1553-45f9-b92b-c4b9d062fff3 None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:19.369454 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b78f78-1553-45f9-b92b-c4b9d062fff3 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:19.369735 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b78f78-1553-45f9-b92b-c4b9d062fff3 None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:19.371862 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 138/285] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jun 23 15:47:19 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 15:47:19.381896 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b918f891-5d0a-4895-9fbf-0b78eadbe6e0 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:19.382184 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b918f891-5d0a-4895-9fbf-0b78eadbe6e0 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:19.382458 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b918f891-5d0a-4895-9fbf-0b78eadbe6e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:19.420134 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-b918f891-5d0a-4895-9fbf-0b78eadbe6e0 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:47:19.421251 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-b918f891-5d0a-4895-9fbf-0b78eadbe6e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:47:19.442623 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d93cfaf-c009-41d4-980f-93bfbe55fc31 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:19.473536 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:19.489362 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63301993-1334-43bf-8809-f4d1b262b811 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:19.490263 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63301993-1334-43bf-8809-f4d1b262b811 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:19.490550 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63301993-1334-43bf-8809-f4d1b262b811 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:19.490872 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63301993-1334-43bf-8809-f4d1b262b811 None admin] PATH_INFO: `/v3/projects/2240bbdc36ed4e59a7454b287b5891ab` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:19.493036 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63301993-1334-43bf-8809-f4d1b262b811 None admin] RBAC: Authorizing `identity:get_project(project_id=2240bbdc36ed4e59a7454b287b5891ab)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:19.494525 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63301993-1334-43bf-8809-f4d1b262b811 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:19.496472 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 148/286] 10.222.0.46 () {56 vars in 1186 bytes} [Thu Jun 23 15:47:19 2022] GET /identity/v3/projects/2240bbdc36ed4e59a7454b287b5891ab => generated 302 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:47:19.503897 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63301993-1334-43bf-8809-f4d1b262b811 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:19.565814 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 139/287] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jun 23 15:47:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 15:47:19.604191 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b918f891-5d0a-4895-9fbf-0b78eadbe6e0 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:19.623049 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:19.693908 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63301993-1334-43bf-8809-f4d1b262b811 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:19.754086 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b918f891-5d0a-4895-9fbf-0b78eadbe6e0 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:19.783858 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a393a1-90e0-4484-a4e2-798559b9e390 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:19.785154 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a393a1-90e0-4484-a4e2-798559b9e390 None admin] REQUEST_METHOD: `PUT` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:19.785639 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a393a1-90e0-4484-a4e2-798559b9e390 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:19.788020 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a393a1-90e0-4484-a4e2-798559b9e390 None admin] PATH_INFO: `/v3/projects/2240bbdc36ed4e59a7454b287b5891ab/users/648dab2f743c4481befe6a87f21263d3/roles/38bbc1cebab84cac9b213ffba44bb4a6` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:19.794708 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a393a1-90e0-4484-a4e2-798559b9e390 None admin] RBAC: Authorizing `identity:create_grant(project_id=2240bbdc36ed4e59a7454b287b5891ab, user_id=648dab2f743c4481befe6a87f21263d3, role_id=38bbc1cebab84cac9b213ffba44bb4a6)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:19.797671 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a393a1-90e0-4484-a4e2-798559b9e390 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:19.823796 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 149/288] 10.222.0.46 () {58 vars in 1439 bytes} [Thu Jun 23 15:47:19 2022] PUT /identity/v3/projects/2240bbdc36ed4e59a7454b287b5891ab/users/648dab2f743c4481befe6a87f21263d3/roles/38bbc1cebab84cac9b213ffba44bb4a6 => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 15:47:19.828405 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c665294b-37a7-4794-b0c4-4d8c5db1620b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:19.829607 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c665294b-37a7-4794-b0c4-4d8c5db1620b None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:19.830105 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c665294b-37a7-4794-b0c4-4d8c5db1620b None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:19.830634 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c665294b-37a7-4794-b0c4-4d8c5db1620b None admin] PATH_INFO: `/v3/roles/member` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:19.844089 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c665294b-37a7-4794-b0c4-4d8c5db1620b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 15:47:19.857659 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-c665294b-37a7-4794-b0c4-4d8c5db1620b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 23 15:47:19.857659 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:47:19.857659 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:47:19.857659 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:47:19.857659 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:47:19.857659 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:47:19.857659 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:47:19.857659 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:47:19.857659 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:47:19.857659 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:47:19.857659 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:47:19.857659 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:47:19.857659 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 15:47:19.857659 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 15:47:19.857659 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 15:47:19.857659 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 15:47:19.858607 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 15:47:19.858607 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 15:47:19.858607 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 15:47:19.858607 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 15:47:19.858607 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:47:19.858607 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:47:19.858607 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:47:19.858607 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:47:19.858607 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:47:19.858607 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:47:19.858607 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:47:19.858607 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Jun 23 15:47:19.858607 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:47:19.858607 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:47:19.858607 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:47:19.858607 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:47:19.859438 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:47:19.859438 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:47:19.859438 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:47:19.859438 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:47:19.859438 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 15:47:19.859438 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 15:47:19.859438 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 15:47:19.859438 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 15:47:19.859438 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 15:47:19.859438 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 15:47:19.859438 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 23 15:47:19.859438 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Jun 23 15:47:19.862179 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 140/289] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Jun 23 15:47:19 2022] GET /identity/v3/roles/member => generated 84 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 15:47:19.868031 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a393a1-90e0-4484-a4e2-798559b9e390 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:19.961954 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:19.998054 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a393a1-90e0-4484-a4e2-798559b9e390 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:20.036776 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b492b3a5-59e1-4fc5-83b9-7defc27ca9dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:20.037669 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b492b3a5-59e1-4fc5-83b9-7defc27ca9dd None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:20.038060 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b492b3a5-59e1-4fc5-83b9-7defc27ca9dd None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:20.038446 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b492b3a5-59e1-4fc5-83b9-7defc27ca9dd None admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:20.039934 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b492b3a5-59e1-4fc5-83b9-7defc27ca9dd None admin] RBAC: Adding query filter params (name=member) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:47:20.040359 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b492b3a5-59e1-4fc5-83b9-7defc27ca9dd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:20.041918 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b492b3a5-59e1-4fc5-83b9-7defc27ca9dd None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:20.052205 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 150/290] 10.222.0.46 () {56 vars in 1101 bytes} [Thu Jun 23 15:47:19 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:47:20.059834 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c665294b-37a7-4794-b0c4-4d8c5db1620b None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:20.150088 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:20.192582 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c665294b-37a7-4794-b0c4-4d8c5db1620b None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:20.247457 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ae5460-1c2f-46ae-8a68-c0102b00adc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:20.248828 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ae5460-1c2f-46ae-8a68-c0102b00adc5 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:20.249314 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ae5460-1c2f-46ae-8a68-c0102b00adc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:20.249760 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ae5460-1c2f-46ae-8a68-c0102b00adc5 None admin] PATH_INFO: `/v3/users/8acd9db80c3b4c03af741ef235678038` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:20.255653 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ae5460-1c2f-46ae-8a68-c0102b00adc5 None admin] RBAC: Authorizing `identity:get_user(user_id=8acd9db80c3b4c03af741ef235678038)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:20.257418 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ae5460-1c2f-46ae-8a68-c0102b00adc5 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:20.259902 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 141/291] 10.222.0.46 () {56 vars in 1177 bytes} [Thu Jun 23 15:47:20 2022] GET /identity/v3/users/8acd9db80c3b4c03af741ef235678038 => generated 297 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:47:20.265537 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b492b3a5-59e1-4fc5-83b9-7defc27ca9dd None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:20.428702 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b492b3a5-59e1-4fc5-83b9-7defc27ca9dd None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:20.457991 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:20.482054 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b692894-e9d1-4682-8696-d9f6e7642fa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:20.482755 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b692894-e9d1-4682-8696-d9f6e7642fa3 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:20.483047 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b692894-e9d1-4682-8696-d9f6e7642fa3 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:20.483323 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b692894-e9d1-4682-8696-d9f6e7642fa3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:20.583510 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b692894-e9d1-4682-8696-d9f6e7642fa3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:20.585007 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b692894-e9d1-4682-8696-d9f6e7642fa3 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:20.727514 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 151/292] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Jun 23 15:47:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 462 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 15:47:20.732844 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ae5460-1c2f-46ae-8a68-c0102b00adc5 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:20.875046 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ae5460-1c2f-46ae-8a68-c0102b00adc5 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:20.936539 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234c38a8-d13a-4668-89d2-c9e68f803816 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:20.937825 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234c38a8-d13a-4668-89d2-c9e68f803816 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:20.938315 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234c38a8-d13a-4668-89d2-c9e68f803816 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:20.938850 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234c38a8-d13a-4668-89d2-c9e68f803816 None admin] PATH_INFO: `/v3/projects/2240bbdc36ed4e59a7454b287b5891ab` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:20.940447 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234c38a8-d13a-4668-89d2-c9e68f803816 None admin] RBAC: Authorizing `identity:get_project(project_id=2240bbdc36ed4e59a7454b287b5891ab)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:20.942189 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234c38a8-d13a-4668-89d2-c9e68f803816 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:20.944241 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 142/293] 10.222.0.46 () {56 vars in 1186 bytes} [Thu Jun 23 15:47:20 2022] GET /identity/v3/projects/2240bbdc36ed4e59a7454b287b5891ab => generated 302 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 15:47:20.950276 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b692894-e9d1-4682-8696-d9f6e7642fa3 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:20.982728 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:21.098554 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b692894-e9d1-4682-8696-d9f6e7642fa3 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:21.142916 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c37391-ccd1-443d-b564-c3c2b6939881 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:21.143811 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c37391-ccd1-443d-b564-c3c2b6939881 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:21.144257 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c37391-ccd1-443d-b564-c3c2b6939881 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:21.144679 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c37391-ccd1-443d-b564-c3c2b6939881 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:21.145683 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c37391-ccd1-443d-b564-c3c2b6939881 None admin] RBAC: Adding query filter params (role.id=e2a5bbf3200a4d4091d72bc7ff78c34d, scope.project.id=2240bbdc36ed4e59a7454b287b5891ab, user.id=8acd9db80c3b4c03af741ef235678038) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:47:21.146145 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c37391-ccd1-443d-b564-c3c2b6939881 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:21.147776 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c37391-ccd1-443d-b564-c3c2b6939881 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:21.159501 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 152/294] 10.222.0.46 () {56 vars in 1374 bytes} [Thu Jun 23 15:47:20 2022] GET /identity/v3/role_assignments?user.id=8acd9db80c3b4c03af741ef235678038&scope.project.id=2240bbdc36ed4e59a7454b287b5891ab&role.id=e2a5bbf3200a4d4091d72bc7ff78c34d => generated 608 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:47:21.532312 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b3753f-065b-4056-ab19-848bc387d161 None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:21.532831 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b3753f-065b-4056-ab19-848bc387d161 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:21.532831 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b3753f-065b-4056-ab19-848bc387d161 None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:21.534506 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 143/295] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jun 23 15:47:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 15:47:21.540668 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d542e66-0e1f-44ff-a944-261ff7569c3a None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:21.540963 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d542e66-0e1f-44ff-a944-261ff7569c3a None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:21.541245 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d542e66-0e1f-44ff-a944-261ff7569c3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:21.559046 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-8d542e66-0e1f-44ff-a944-261ff7569c3a None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:47:21.560267 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-8d542e66-0e1f-44ff-a944-261ff7569c3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:47:21.651277 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 153/296] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jun 23 15:47:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 15:47:21.670475 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f520b893-5bcc-49e6-be48-b83f8d36cd59 None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:21.670808 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f520b893-5bcc-49e6-be48-b83f8d36cd59 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:21.671070 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f520b893-5bcc-49e6-be48-b83f8d36cd59 None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:21.672727 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 144/297] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jun 23 15:47:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 15:47:21.678035 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73eb89e2-bd30-43f1-902a-12b129f81b6a None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:21.678320 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73eb89e2-bd30-43f1-902a-12b129f81b6a None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:21.678604 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73eb89e2-bd30-43f1-902a-12b129f81b6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:21.685718 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d194c9a-f215-432b-984c-256c2c0ade20 None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:21.686009 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d194c9a-f215-432b-984c-256c2c0ade20 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:21.686291 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d194c9a-f215-432b-984c-256c2c0ade20 None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:21.689244 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 145/298] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jun 23 15:47:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 15:47:21.691270 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d194c9a-f215-432b-984c-256c2c0ade20 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:21.705627 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-73eb89e2-bd30-43f1-902a-12b129f81b6a None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:47:21.707383 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:21.707592 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-73eb89e2-bd30-43f1-902a-12b129f81b6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:47:21.723928 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:21.800603 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 154/299] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jun 23 15:47:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 15:47:21.810713 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26390fbd-7c02-4cf6-b5b5-67672e3c3e25 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:21.810713 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26390fbd-7c02-4cf6-b5b5-67672e3c3e25 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:21.810713 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26390fbd-7c02-4cf6-b5b5-67672e3c3e25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:21.822017 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-26390fbd-7c02-4cf6-b5b5-67672e3c3e25 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:47:21.823437 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-26390fbd-7c02-4cf6-b5b5-67672e3c3e25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:47:21.852958 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d194c9a-f215-432b-984c-256c2c0ade20 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:21.894946 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9c825f-f310-4c3a-8402-6b4546d4a183 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:21.895766 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9c825f-f310-4c3a-8402-6b4546d4a183 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:21.896048 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9c825f-f310-4c3a-8402-6b4546d4a183 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:21.896322 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9c825f-f310-4c3a-8402-6b4546d4a183 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:21.899631 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 155/300] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jun 23 15:47:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 15:47:21.900716 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26390fbd-7c02-4cf6-b5b5-67672e3c3e25 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:21.912353 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9c825f-f310-4c3a-8402-6b4546d4a183 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 15:47:21.920787 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-7a9c825f-f310-4c3a-8402-6b4546d4a183 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 23 15:47:21.920787 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:47:21.920787 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:47:21.920787 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:47:21.920787 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:47:21.920787 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:47:21.920787 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:47:21.920787 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:47:21.920787 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:47:21.920787 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:47:21.920787 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:47:21.920787 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:47:21.920787 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 15:47:21.920787 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 15:47:21.920787 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 15:47:21.920787 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 15:47:21.921715 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 15:47:21.921715 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 15:47:21.921715 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 15:47:21.921715 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 15:47:21.921715 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:47:21.921715 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:47:21.921715 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:47:21.921715 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:47:21.921715 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:47:21.921715 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:47:21.921715 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:47:21.921715 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Jun 23 15:47:21.921715 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:47:21.921715 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:47:21.921715 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:47:21.921715 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:47:21.922521 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:47:21.922521 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:47:21.922521 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:47:21.922521 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:47:21.922521 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 15:47:21.922521 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 15:47:21.922521 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 15:47:21.922521 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 15:47:21.922521 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 15:47:21.922521 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 15:47:21.922521 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 23 15:47:21.922521 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Jun 23 15:47:21.930152 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 146/301] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Jun 23 15:47:21 2022] GET /identity/v3/roles/reader => generated 84 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:47:21.931422 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9c825f-f310-4c3a-8402-6b4546d4a183 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:21.982080 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:22.004558 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:22.013095 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26390fbd-7c02-4cf6-b5b5-67672e3c3e25 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:22.048819 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9c825f-f310-4c3a-8402-6b4546d4a183 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:22.053409 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07841c24-0eb2-4cf4-a865-6d724ff6bf91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:22.054503 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07841c24-0eb2-4cf4-a865-6d724ff6bf91 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:22.055612 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07841c24-0eb2-4cf4-a865-6d724ff6bf91 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:22.056066 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07841c24-0eb2-4cf4-a865-6d724ff6bf91 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:22.057695 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07841c24-0eb2-4cf4-a865-6d724ff6bf91 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:22.059559 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07841c24-0eb2-4cf4-a865-6d724ff6bf91 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:22.061725 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 156/302] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Jun 23 15:47:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:47:22.061725 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07841c24-0eb2-4cf4-a865-6d724ff6bf91 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:22.093606 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947a3f94-b9bf-489a-adce-e5784af88541 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:22.094286 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947a3f94-b9bf-489a-adce-e5784af88541 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:22.094569 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947a3f94-b9bf-489a-adce-e5784af88541 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:22.094867 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947a3f94-b9bf-489a-adce-e5784af88541 None admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:22.095629 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947a3f94-b9bf-489a-adce-e5784af88541 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:47:22.095939 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947a3f94-b9bf-489a-adce-e5784af88541 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:22.097323 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947a3f94-b9bf-489a-adce-e5784af88541 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:22.111196 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 147/303] 10.222.0.46 () {56 vars in 1101 bytes} [Thu Jun 23 15:47:21 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:47:22.111196 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947a3f94-b9bf-489a-adce-e5784af88541 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:22.177054 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07841c24-0eb2-4cf4-a865-6d724ff6bf91 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:22.226250 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947a3f94-b9bf-489a-adce-e5784af88541 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:22.228567 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cd019e-5cf4-482a-95ec-1f8a1146e41a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:22.229456 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cd019e-5cf4-482a-95ec-1f8a1146e41a None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:22.229748 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cd019e-5cf4-482a-95ec-1f8a1146e41a None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:22.230028 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cd019e-5cf4-482a-95ec-1f8a1146e41a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:22.241203 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0cd019e-5cf4-482a-95ec-1f8a1146e41a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 15:47:22.249943 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-f0cd019e-5cf4-482a-95ec-1f8a1146e41a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 23 15:47:22.249943 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:47:22.249943 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:47:22.249943 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:47:22.249943 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:47:22.249943 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:47:22.249943 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:47:22.249943 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:47:22.249943 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:47:22.249943 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:47:22.249943 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:47:22.249943 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:47:22.249943 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 15:47:22.249943 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 15:47:22.249943 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 15:47:22.249943 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 15:47:22.250873 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 15:47:22.250873 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 15:47:22.250873 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 15:47:22.250873 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 15:47:22.250873 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:47:22.250873 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:47:22.250873 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:47:22.250873 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:47:22.250873 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:47:22.250873 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:47:22.250873 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:47:22.250873 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Jun 23 15:47:22.250873 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:47:22.250873 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:47:22.250873 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:47:22.250873 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:47:22.251704 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:47:22.251704 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:47:22.251704 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:47:22.251704 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:47:22.251704 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 15:47:22.251704 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 15:47:22.251704 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 15:47:22.251704 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 15:47:22.251704 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 15:47:22.251704 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 15:47:22.251704 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 23 15:47:22.251704 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Jun 23 15:47:22.251704 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 157/304] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Jun 23 15:47:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:47:22.252430 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cd019e-5cf4-482a-95ec-1f8a1146e41a None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:22.254759 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:22.267910 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:22.276480 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0eabf95-f4ea-4507-9939-31000ec6b4e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:22.277484 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0eabf95-f4ea-4507-9939-31000ec6b4e0 None admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:22.277964 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0eabf95-f4ea-4507-9939-31000ec6b4e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:22.278408 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0eabf95-f4ea-4507-9939-31000ec6b4e0 None admin] PATH_INFO: `/v3/groups` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:22.279334 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0eabf95-f4ea-4507-9939-31000ec6b4e0 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:22.280950 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0eabf95-f4ea-4507-9939-31000ec6b4e0 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:22.297650 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 148/305] 10.222.0.46 () {60 vars in 1134 bytes} [Thu Jun 23 15:47:22 2022] POST /identity/v3/groups => generated 229 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 15:47:22.299010 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0eabf95-f4ea-4507-9939-31000ec6b4e0 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:22.399984 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cd019e-5cf4-482a-95ec-1f8a1146e41a None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:22.445267 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72eafaaa-0381-418e-a8ef-d5ae0dc2e379 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:22.446170 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72eafaaa-0381-418e-a8ef-d5ae0dc2e379 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:22.446457 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72eafaaa-0381-418e-a8ef-d5ae0dc2e379 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:22.446759 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72eafaaa-0381-418e-a8ef-d5ae0dc2e379 None admin] PATH_INFO: `/v3/users/509c2a6e8deb48fcbcacd1f6319d6f8b` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:22.455006 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0eabf95-f4ea-4507-9939-31000ec6b4e0 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:22.475141 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72eafaaa-0381-418e-a8ef-d5ae0dc2e379 None admin] RBAC: Authorizing `identity:get_user(user_id=509c2a6e8deb48fcbcacd1f6319d6f8b)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:22.477159 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72eafaaa-0381-418e-a8ef-d5ae0dc2e379 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:22.482030 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 158/306] 10.222.0.46 () {56 vars in 1177 bytes} [Thu Jun 23 15:47:22 2022] GET /identity/v3/users/509c2a6e8deb48fcbcacd1f6319d6f8b => generated 293 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:47:22.482030 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dc74f1-ba81-45cc-81b2-a9279ce90a40 None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:22.482030 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dc74f1-ba81-45cc-81b2-a9279ce90a40 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:22.482030 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dc74f1-ba81-45cc-81b2-a9279ce90a40 None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:22.482030 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 159/307] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jun 23 15:47:22 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 15:47:22.486573 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4dc74f1-ba81-45cc-81b2-a9279ce90a40 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:22.503671 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e02c320-0298-4746-876b-ec3de3ef8d8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:22.504396 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e02c320-0298-4746-876b-ec3de3ef8d8e None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:22.504686 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e02c320-0298-4746-876b-ec3de3ef8d8e None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:22.504967 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e02c320-0298-4746-876b-ec3de3ef8d8e None admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:22.505730 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e02c320-0298-4746-876b-ec3de3ef8d8e None admin] RBAC: Adding query filter params (name=member) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:47:22.506046 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e02c320-0298-4746-876b-ec3de3ef8d8e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:22.507476 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e02c320-0298-4746-876b-ec3de3ef8d8e None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:22.516936 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 149/308] 10.222.0.46 () {56 vars in 1101 bytes} [Thu Jun 23 15:47:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:47:22.518075 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543ea274-659f-460d-a744-1b2504975d46 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:22.518375 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543ea274-659f-460d-a744-1b2504975d46 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:22.519259 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543ea274-659f-460d-a744-1b2504975d46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:22.536664 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-543ea274-659f-460d-a744-1b2504975d46 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:47:22.538641 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-543ea274-659f-460d-a744-1b2504975d46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:47:22.577962 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:22.598349 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:22.622019 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 150/309] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jun 23 15:47:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 15:47:22.622909 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543ea274-659f-460d-a744-1b2504975d46 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:22.624066 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4dc74f1-ba81-45cc-81b2-a9279ce90a40 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:22.662580 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150a1244-bd1f-4ee9-847d-f6107dd6ed7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:22.663505 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150a1244-bd1f-4ee9-847d-f6107dd6ed7e None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:22.664471 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150a1244-bd1f-4ee9-847d-f6107dd6ed7e None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:22.664889 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150a1244-bd1f-4ee9-847d-f6107dd6ed7e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:22.665866 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150a1244-bd1f-4ee9-847d-f6107dd6ed7e None admin] RBAC: Adding query filter params (role.id=38bbc1cebab84cac9b213ffba44bb4a6, user.id=509c2a6e8deb48fcbcacd1f6319d6f8b, scope.system=all) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:47:22.666334 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150a1244-bd1f-4ee9-847d-f6107dd6ed7e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:22.667959 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150a1244-bd1f-4ee9-847d-f6107dd6ed7e None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:22.673747 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 160/310] 10.222.0.46 () {56 vars in 1308 bytes} [Thu Jun 23 15:47:22 2022] GET /identity/v3/role_assignments?user.id=509c2a6e8deb48fcbcacd1f6319d6f8b&scope.system=all&role.id=38bbc1cebab84cac9b213ffba44bb4a6 => generated 226 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:47:22.680402 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150a1244-bd1f-4ee9-847d-f6107dd6ed7e None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:22.753763 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543ea274-659f-460d-a744-1b2504975d46 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:22.801581 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9ad584-307e-4da3-a70e-c31c3b795da4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:22.803253 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9ad584-307e-4da3-a70e-c31c3b795da4 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:22.803757 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9ad584-307e-4da3-a70e-c31c3b795da4 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:22.804238 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9ad584-307e-4da3-a70e-c31c3b795da4 None admin] PATH_INFO: `/v3/users/47443d2110cd4bfb83a197c0664325a3` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:22.805969 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9ad584-307e-4da3-a70e-c31c3b795da4 None admin] RBAC: Authorizing `identity:get_user(user_id=47443d2110cd4bfb83a197c0664325a3)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:22.807998 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9ad584-307e-4da3-a70e-c31c3b795da4 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:22.810357 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 151/311] 10.222.0.46 () {56 vars in 1177 bytes} [Thu Jun 23 15:47:22 2022] GET /identity/v3/users/47443d2110cd4bfb83a197c0664325a3 => generated 293 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:47:22.812967 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150a1244-bd1f-4ee9-847d-f6107dd6ed7e None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:22.820515 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9ad584-307e-4da3-a70e-c31c3b795da4 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:22.847905 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:22.872135 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd627ea-840d-4089-8e71-8763ba793ae6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:22.873675 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd627ea-840d-4089-8e71-8763ba793ae6 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:22.874064 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd627ea-840d-4089-8e71-8763ba793ae6 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:22.874445 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd627ea-840d-4089-8e71-8763ba793ae6 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:22.884759 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:22.892115 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd627ea-840d-4089-8e71-8763ba793ae6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 15:47:22.904074 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-5cd627ea-840d-4089-8e71-8763ba793ae6 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 23 15:47:22.904074 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:47:22.904074 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:47:22.904074 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:47:22.904074 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:47:22.904074 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:47:22.904074 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:47:22.904074 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:47:22.904074 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:47:22.904074 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:47:22.904074 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:47:22.904074 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:47:22.904074 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 15:47:22.904074 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 15:47:22.904074 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 15:47:22.904074 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 15:47:22.905170 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 15:47:22.905170 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 15:47:22.905170 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 15:47:22.905170 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 15:47:22.905170 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:47:22.905170 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:47:22.905170 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:47:22.905170 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:47:22.905170 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:47:22.905170 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:47:22.905170 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:47:22.905170 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Jun 23 15:47:22.905170 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:47:22.905170 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:47:22.905170 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:47:22.905170 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:47:22.905976 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:47:22.905976 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:47:22.905976 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:47:22.905976 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:47:22.905976 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 15:47:22.905976 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 15:47:22.905976 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 15:47:22.905976 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 15:47:22.905976 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 15:47:22.905976 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 15:47:22.905976 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 23 15:47:22.905976 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Jun 23 15:47:22.906575 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 161/312] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Jun 23 15:47:22 2022] GET /identity/v3/roles/reader => generated 84 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:47:22.913049 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd627ea-840d-4089-8e71-8763ba793ae6 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:22.953932 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9ad584-307e-4da3-a70e-c31c3b795da4 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:22.992848 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0102bd88-d050-4110-8960-a7d15c838bed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:22.994075 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0102bd88-d050-4110-8960-a7d15c838bed None admin] REQUEST_METHOD: `PUT` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:22.994559 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0102bd88-d050-4110-8960-a7d15c838bed None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:22.995075 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0102bd88-d050-4110-8960-a7d15c838bed None admin] PATH_INFO: `/v3/system/users/47443d2110cd4bfb83a197c0664325a3/roles/e2a5bbf3200a4d4091d72bc7ff78c34d` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:22.996968 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0102bd88-d050-4110-8960-a7d15c838bed None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=47443d2110cd4bfb83a197c0664325a3, role_id=e2a5bbf3200a4d4091d72bc7ff78c34d)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:22.998592 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0102bd88-d050-4110-8960-a7d15c838bed None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:23.009735 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 152/313] 10.222.0.46 () {58 vars in 1334 bytes} [Thu Jun 23 15:47:22 2022] PUT /identity/v3/system/users/47443d2110cd4bfb83a197c0664325a3/roles/e2a5bbf3200a4d4091d72bc7ff78c34d => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 15:47:23.047731 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd627ea-840d-4089-8e71-8763ba793ae6 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:23.090581 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0077a02-2856-4688-8c19-703dc6371fb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:23.091908 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0077a02-2856-4688-8c19-703dc6371fb1 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:23.092467 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0077a02-2856-4688-8c19-703dc6371fb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:23.092961 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0077a02-2856-4688-8c19-703dc6371fb1 None admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:23.093960 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0077a02-2856-4688-8c19-703dc6371fb1 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:47:23.094481 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0077a02-2856-4688-8c19-703dc6371fb1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:23.096176 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0077a02-2856-4688-8c19-703dc6371fb1 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:23.109260 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 162/314] 10.222.0.46 () {56 vars in 1101 bytes} [Thu Jun 23 15:47:22 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:47:23.115491 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0102bd88-d050-4110-8960-a7d15c838bed None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:23.231449 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0102bd88-d050-4110-8960-a7d15c838bed None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:23.248047 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:23.271179 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e529811-57e5-4eb8-ad51-9a10e63cdd15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:23.272076 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e529811-57e5-4eb8-ad51-9a10e63cdd15 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:23.272426 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e529811-57e5-4eb8-ad51-9a10e63cdd15 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:23.272707 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e529811-57e5-4eb8-ad51-9a10e63cdd15 None admin] PATH_INFO: `/v3/users/648dab2f743c4481befe6a87f21263d3` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:23.273999 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e529811-57e5-4eb8-ad51-9a10e63cdd15 None admin] RBAC: Authorizing `identity:get_user(user_id=648dab2f743c4481befe6a87f21263d3)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:23.275532 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e529811-57e5-4eb8-ad51-9a10e63cdd15 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:23.277814 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 153/315] 10.222.0.46 () {56 vars in 1177 bytes} [Thu Jun 23 15:47:23 2022] GET /identity/v3/users/648dab2f743c4481befe6a87f21263d3 => generated 297 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:47:23.283475 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0077a02-2856-4688-8c19-703dc6371fb1 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:23.336351 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:23.400192 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0077a02-2856-4688-8c19-703dc6371fb1 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:23.437317 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601f3631-e2ce-4885-8d29-5687c430d480 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:23.437979 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601f3631-e2ce-4885-8d29-5687c430d480 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:23.438271 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601f3631-e2ce-4885-8d29-5687c430d480 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:23.438551 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601f3631-e2ce-4885-8d29-5687c430d480 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:23.515423 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601f3631-e2ce-4885-8d29-5687c430d480 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:23.516806 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601f3631-e2ce-4885-8d29-5687c430d480 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:23.566135 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:23.623343 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 163/316] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Jun 23 15:47:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 15:47:23.628734 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e529811-57e5-4eb8-ad51-9a10e63cdd15 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:23.733488 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e529811-57e5-4eb8-ad51-9a10e63cdd15 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:23.769684 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4835cae-9b6c-457e-85dc-1f072b02b5d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:23.770491 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4835cae-9b6c-457e-85dc-1f072b02b5d5 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:23.770808 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4835cae-9b6c-457e-85dc-1f072b02b5d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:23.771091 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4835cae-9b6c-457e-85dc-1f072b02b5d5 None admin] PATH_INFO: `/v3/projects/2240bbdc36ed4e59a7454b287b5891ab` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:23.772315 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4835cae-9b6c-457e-85dc-1f072b02b5d5 None admin] RBAC: Authorizing `identity:get_project(project_id=2240bbdc36ed4e59a7454b287b5891ab)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:23.773963 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4835cae-9b6c-457e-85dc-1f072b02b5d5 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:23.776383 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 154/317] 10.222.0.46 () {56 vars in 1186 bytes} [Thu Jun 23 15:47:23 2022] GET /identity/v3/projects/2240bbdc36ed4e59a7454b287b5891ab => generated 302 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:47:23.782243 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601f3631-e2ce-4885-8d29-5687c430d480 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:23.888148 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601f3631-e2ce-4885-8d29-5687c430d480 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:23.923385 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3478d9a5-9953-462c-864e-074e12eb975e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:23.924048 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3478d9a5-9953-462c-864e-074e12eb975e None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:23.924332 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3478d9a5-9953-462c-864e-074e12eb975e None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:23.924608 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3478d9a5-9953-462c-864e-074e12eb975e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:23.925447 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3478d9a5-9953-462c-864e-074e12eb975e None admin] RBAC: Adding query filter params (role.id=38bbc1cebab84cac9b213ffba44bb4a6, scope.project.id=2240bbdc36ed4e59a7454b287b5891ab, user.id=648dab2f743c4481befe6a87f21263d3) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:47:23.925761 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3478d9a5-9953-462c-864e-074e12eb975e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:23.927250 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3478d9a5-9953-462c-864e-074e12eb975e None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:23.934117 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 164/318] 10.222.0.46 () {56 vars in 1374 bytes} [Thu Jun 23 15:47:23 2022] GET /identity/v3/role_assignments?user.id=648dab2f743c4481befe6a87f21263d3&scope.project.id=2240bbdc36ed4e59a7454b287b5891ab&role.id=38bbc1cebab84cac9b213ffba44bb4a6 => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:47:24.475008 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da30845-3a16-443f-a04b-e1f3fb5685cf None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:24.475390 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da30845-3a16-443f-a04b-e1f3fb5685cf None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:24.475565 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da30845-3a16-443f-a04b-e1f3fb5685cf None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:24.477671 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 155/319] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jun 23 15:47:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 15:47:24.482719 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e569de7b-3372-40c2-8296-466b3d0e9708 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:24.483165 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e569de7b-3372-40c2-8296-466b3d0e9708 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:24.483329 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e569de7b-3372-40c2-8296-466b3d0e9708 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:24.501554 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-e569de7b-3372-40c2-8296-466b3d0e9708 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:47:24.502834 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-e569de7b-3372-40c2-8296-466b3d0e9708 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:47:24.507278 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:24.586729 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 165/320] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jun 23 15:47:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 15:47:24.610285 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da30845-3a16-443f-a04b-e1f3fb5685cf None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:24.663412 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:24.717655 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da30845-3a16-443f-a04b-e1f3fb5685cf None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:24.752725 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b56b296-3c0a-4130-84d0-47d72aece601 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:24.753456 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b56b296-3c0a-4130-84d0-47d72aece601 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:24.753747 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b56b296-3c0a-4130-84d0-47d72aece601 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:24.754053 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b56b296-3c0a-4130-84d0-47d72aece601 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:24.755250 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b56b296-3c0a-4130-84d0-47d72aece601 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:24.756715 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b56b296-3c0a-4130-84d0-47d72aece601 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:24.759106 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 156/321] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Jun 23 15:47:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:47:24.762296 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4c038a-fbbc-4adb-86cb-43258f8cdd74 None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:24.762590 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4c038a-fbbc-4adb-86cb-43258f8cdd74 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:24.762923 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4c038a-fbbc-4adb-86cb-43258f8cdd74 None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:24.764179 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b56b296-3c0a-4130-84d0-47d72aece601 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:24.764580 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 166/322] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jun 23 15:47:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 15:47:24.769518 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec64d505-234a-4299-b10a-1249ef9cc575 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:24.769810 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec64d505-234a-4299-b10a-1249ef9cc575 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:24.770093 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec64d505-234a-4299-b10a-1249ef9cc575 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:24.787895 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-ec64d505-234a-4299-b10a-1249ef9cc575 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:47:24.789167 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-ec64d505-234a-4299-b10a-1249ef9cc575 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:47:24.871743 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 167/323] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jun 23 15:47:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 15:47:24.887813 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b56b296-3c0a-4130-84d0-47d72aece601 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:24.894277 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec64d505-234a-4299-b10a-1249ef9cc575 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:24.907440 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:24.926681 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348e75e2-6883-49f6-ade9-052f47580b65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:24.927388 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348e75e2-6883-49f6-ade9-052f47580b65 None admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:24.927677 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348e75e2-6883-49f6-ade9-052f47580b65 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:24.927954 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348e75e2-6883-49f6-ade9-052f47580b65 None admin] PATH_INFO: `/v3/groups` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:24.928667 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348e75e2-6883-49f6-ade9-052f47580b65 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:24.930875 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348e75e2-6883-49f6-ade9-052f47580b65 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:24.947278 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 157/324] 10.222.0.46 () {60 vars in 1134 bytes} [Thu Jun 23 15:47:24 2022] POST /identity/v3/groups => generated 226 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 15:47:24.973322 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:25.012716 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec64d505-234a-4299-b10a-1249ef9cc575 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:25.051359 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7635919b-b170-47f3-8a44-0623eb063180 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:25.052239 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7635919b-b170-47f3-8a44-0623eb063180 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:25.052526 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7635919b-b170-47f3-8a44-0623eb063180 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:25.052817 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7635919b-b170-47f3-8a44-0623eb063180 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:25.066572 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7635919b-b170-47f3-8a44-0623eb063180 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 15:47:25.075186 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-7635919b-b170-47f3-8a44-0623eb063180 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 23 15:47:25.075186 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:47:25.075186 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:47:25.075186 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:47:25.075186 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:47:25.075186 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:47:25.075186 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:47:25.075186 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:47:25.075186 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:47:25.075186 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:47:25.075186 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:47:25.075186 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:47:25.075186 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 15:47:25.075186 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 15:47:25.075186 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 15:47:25.075186 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 15:47:25.076195 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 15:47:25.076195 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 15:47:25.076195 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 15:47:25.076195 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 15:47:25.076195 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:47:25.076195 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:47:25.076195 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:47:25.076195 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:47:25.076195 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:47:25.076195 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:47:25.076195 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:47:25.076195 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Jun 23 15:47:25.076195 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:47:25.076195 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:47:25.076195 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:47:25.076195 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:47:25.077007 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:47:25.077007 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:47:25.077007 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:47:25.077007 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:47:25.077007 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 15:47:25.077007 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 15:47:25.077007 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 15:47:25.077007 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 15:47:25.077007 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 15:47:25.077007 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 15:47:25.077007 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 23 15:47:25.077007 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Jun 23 15:47:25.077007 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 168/325] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Jun 23 15:47:24 2022] GET /identity/v3/roles/reader => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 15:47:25.084309 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348e75e2-6883-49f6-ade9-052f47580b65 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:25.111599 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85b8f57-f418-4a3a-8536-9afd38ebcbb1 None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:25.112111 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85b8f57-f418-4a3a-8536-9afd38ebcbb1 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:25.112535 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85b8f57-f418-4a3a-8536-9afd38ebcbb1 None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:25.114088 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 169/326] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jun 23 15:47:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 15:47:25.121771 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a280bae0-b93c-492a-9b5f-ba0ab2aca75d None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:25.122053 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a280bae0-b93c-492a-9b5f-ba0ab2aca75d None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:25.122330 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a280bae0-b93c-492a-9b5f-ba0ab2aca75d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:25.140442 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-a280bae0-b93c-492a-9b5f-ba0ab2aca75d None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:47:25.141649 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-a280bae0-b93c-492a-9b5f-ba0ab2aca75d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:47:25.216257 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348e75e2-6883-49f6-ade9-052f47580b65 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:25.235250 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 170/327] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jun 23 15:47:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 15:47:25.262494 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a280bae0-b93c-492a-9b5f-ba0ab2aca75d None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:25.265807 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289108c9-d746-4a61-84e5-9c5ff9381409 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:25.267274 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289108c9-d746-4a61-84e5-9c5ff9381409 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:25.267567 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289108c9-d746-4a61-84e5-9c5ff9381409 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:25.267853 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289108c9-d746-4a61-84e5-9c5ff9381409 None admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:25.268636 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289108c9-d746-4a61-84e5-9c5ff9381409 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:47:25.268956 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289108c9-d746-4a61-84e5-9c5ff9381409 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:25.270499 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289108c9-d746-4a61-84e5-9c5ff9381409 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:25.282376 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 158/328] 10.222.0.46 () {56 vars in 1101 bytes} [Thu Jun 23 15:47:25 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 15:47:25.288450 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289108c9-d746-4a61-84e5-9c5ff9381409 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:25.344337 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:25.348663 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:25.412937 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a280bae0-b93c-492a-9b5f-ba0ab2aca75d None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:25.432859 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289108c9-d746-4a61-84e5-9c5ff9381409 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:25.477487 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd21ad3-300e-4be7-93d0-aac2612564fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:25.478787 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd21ad3-300e-4be7-93d0-aac2612564fe None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:25.479230 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd21ad3-300e-4be7-93d0-aac2612564fe None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:25.479664 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd21ad3-300e-4be7-93d0-aac2612564fe None admin] PATH_INFO: `/v3/roles/member` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:25.481992 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce413b04-dd98-4195-a0d0-2ce01d77e75f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:25.483319 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce413b04-dd98-4195-a0d0-2ce01d77e75f None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:25.483606 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce413b04-dd98-4195-a0d0-2ce01d77e75f None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:25.483884 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce413b04-dd98-4195-a0d0-2ce01d77e75f None admin] PATH_INFO: `/v3/users/509c2a6e8deb48fcbcacd1f6319d6f8b` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:25.485738 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce413b04-dd98-4195-a0d0-2ce01d77e75f None admin] RBAC: Authorizing `identity:get_user(user_id=509c2a6e8deb48fcbcacd1f6319d6f8b)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:25.487259 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce413b04-dd98-4195-a0d0-2ce01d77e75f None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:25.489736 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 159/329] 10.222.0.46 () {56 vars in 1177 bytes} [Thu Jun 23 15:47:25 2022] GET /identity/v3/users/509c2a6e8deb48fcbcacd1f6319d6f8b => generated 293 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:47:25.497450 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce413b04-dd98-4195-a0d0-2ce01d77e75f None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:25.499656 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd21ad3-300e-4be7-93d0-aac2612564fe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 15:47:25.508904 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-2cd21ad3-300e-4be7-93d0-aac2612564fe None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 23 15:47:25.508904 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:47:25.508904 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:47:25.508904 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:47:25.508904 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:47:25.508904 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:47:25.508904 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:47:25.508904 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:47:25.508904 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:47:25.508904 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:47:25.508904 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:47:25.508904 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:47:25.508904 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 15:47:25.508904 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 15:47:25.508904 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 15:47:25.508904 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 15:47:25.509843 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 15:47:25.509843 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 15:47:25.509843 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 15:47:25.509843 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 15:47:25.509843 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:47:25.509843 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:47:25.509843 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:47:25.509843 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:47:25.509843 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:47:25.509843 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:47:25.509843 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:47:25.509843 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Jun 23 15:47:25.509843 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:47:25.509843 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:47:25.509843 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:47:25.509843 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:47:25.510653 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:47:25.510653 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:47:25.510653 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:47:25.510653 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:47:25.510653 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 15:47:25.510653 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 15:47:25.510653 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 15:47:25.510653 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 15:47:25.510653 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 15:47:25.510653 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 15:47:25.510653 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 23 15:47:25.510653 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Jun 23 15:47:25.510653 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 171/330] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Jun 23 15:47:25 2022] GET /identity/v3/roles/member => generated 84 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 15:47:25.517277 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd21ad3-300e-4be7-93d0-aac2612564fe None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:25.652812 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce413b04-dd98-4195-a0d0-2ce01d77e75f None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:25.666565 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:25.688252 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd21ad3-300e-4be7-93d0-aac2612564fe None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:25.703224 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d875cd-a485-4ffe-a458-91fed984cb79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:25.704214 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d875cd-a485-4ffe-a458-91fed984cb79 None admin] REQUEST_METHOD: `PUT` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:25.704565 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d875cd-a485-4ffe-a458-91fed984cb79 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:25.704913 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d875cd-a485-4ffe-a458-91fed984cb79 None admin] PATH_INFO: `/v3/system/users/509c2a6e8deb48fcbcacd1f6319d6f8b/roles/38bbc1cebab84cac9b213ffba44bb4a6` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:25.706674 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d875cd-a485-4ffe-a458-91fed984cb79 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=509c2a6e8deb48fcbcacd1f6319d6f8b, role_id=38bbc1cebab84cac9b213ffba44bb4a6)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:25.708305 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d875cd-a485-4ffe-a458-91fed984cb79 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:25.718713 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 160/331] 10.222.0.46 () {58 vars in 1334 bytes} [Thu Jun 23 15:47:25 2022] PUT /identity/v3/system/users/509c2a6e8deb48fcbcacd1f6319d6f8b/roles/38bbc1cebab84cac9b213ffba44bb4a6 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 15:47:25.719915 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:25.745527 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923dcd9d-80a2-42ed-8b97-5008395a8693 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:25.746248 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923dcd9d-80a2-42ed-8b97-5008395a8693 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:25.746517 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923dcd9d-80a2-42ed-8b97-5008395a8693 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:25.746887 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923dcd9d-80a2-42ed-8b97-5008395a8693 None admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:25.747681 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923dcd9d-80a2-42ed-8b97-5008395a8693 None admin] RBAC: Adding query filter params (name=member) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:47:25.748046 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923dcd9d-80a2-42ed-8b97-5008395a8693 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:25.749581 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923dcd9d-80a2-42ed-8b97-5008395a8693 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:25.760755 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 172/332] 10.222.0.46 () {56 vars in 1101 bytes} [Thu Jun 23 15:47:25 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:47:25.767165 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d875cd-a485-4ffe-a458-91fed984cb79 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:25.889880 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d875cd-a485-4ffe-a458-91fed984cb79 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:25.932326 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b827e4f-99bd-441d-afd9-6d2d8f3df241 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:25.933255 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b827e4f-99bd-441d-afd9-6d2d8f3df241 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:25.933541 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b827e4f-99bd-441d-afd9-6d2d8f3df241 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:25.933820 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b827e4f-99bd-441d-afd9-6d2d8f3df241 None admin] PATH_INFO: `/v3/users/47443d2110cd4bfb83a197c0664325a3` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:25.936355 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b827e4f-99bd-441d-afd9-6d2d8f3df241 None admin] RBAC: Authorizing `identity:get_user(user_id=47443d2110cd4bfb83a197c0664325a3)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:25.937859 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b827e4f-99bd-441d-afd9-6d2d8f3df241 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:25.940147 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 161/333] 10.222.0.46 () {56 vars in 1177 bytes} [Thu Jun 23 15:47:25 2022] GET /identity/v3/users/47443d2110cd4bfb83a197c0664325a3 => generated 293 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:47:25.946399 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923dcd9d-80a2-42ed-8b97-5008395a8693 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:26.064727 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923dcd9d-80a2-42ed-8b97-5008395a8693 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:26.105361 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1624ca12-148b-427a-83c1-e22fcc2e5c62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:26.106039 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1624ca12-148b-427a-83c1-e22fcc2e5c62 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:26.106328 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1624ca12-148b-427a-83c1-e22fcc2e5c62 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:26.106609 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1624ca12-148b-427a-83c1-e22fcc2e5c62 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:26.107510 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1624ca12-148b-427a-83c1-e22fcc2e5c62 None admin] RBAC: Adding query filter params (role.id=e2a5bbf3200a4d4091d72bc7ff78c34d, user.id=47443d2110cd4bfb83a197c0664325a3, scope.system=all) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:47:26.107831 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1624ca12-148b-427a-83c1-e22fcc2e5c62 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:26.109305 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1624ca12-148b-427a-83c1-e22fcc2e5c62 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:26.124632 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 173/334] 10.222.0.46 () {56 vars in 1308 bytes} [Thu Jun 23 15:47:25 2022] GET /identity/v3/role_assignments?user.id=47443d2110cd4bfb83a197c0664325a3&scope.system=all&role.id=e2a5bbf3200a4d4091d72bc7ff78c34d => generated 510 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:47:27.416952 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc69f06d-f73d-41e6-81d9-27fed88d0a00 None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:27.418048 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc69f06d-f73d-41e6-81d9-27fed88d0a00 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:27.418591 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc69f06d-f73d-41e6-81d9-27fed88d0a00 None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:27.420489 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 162/335] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jun 23 15:47:27 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 15:47:27.427037 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd8d0af-d6a5-4cd5-9bab-535e45b9216d None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:27.427949 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd8d0af-d6a5-4cd5-9bab-535e45b9216d None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:27.428424 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd8d0af-d6a5-4cd5-9bab-535e45b9216d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:27.449262 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-dbd8d0af-d6a5-4cd5-9bab-535e45b9216d None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:47:27.450809 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-dbd8d0af-d6a5-4cd5-9bab-535e45b9216d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:47:27.496312 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:27.536292 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 174/336] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jun 23 15:47:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 15:47:27.589619 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc69f06d-f73d-41e6-81d9-27fed88d0a00 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:27.647595 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:27.720748 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc69f06d-f73d-41e6-81d9-27fed88d0a00 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:27.761206 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3714734-3bf3-43e0-ae2d-971d6700655b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:27.762022 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3714734-3bf3-43e0-ae2d-971d6700655b None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:27.762313 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3714734-3bf3-43e0-ae2d-971d6700655b None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:27.762594 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3714734-3bf3-43e0-ae2d-971d6700655b None admin] PATH_INFO: `/v3/roles/member` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:27.783783 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3714734-3bf3-43e0-ae2d-971d6700655b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 15:47:27.792609 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-b3714734-3bf3-43e0-ae2d-971d6700655b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 23 15:47:27.792609 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:47:27.792609 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:47:27.792609 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:47:27.792609 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:47:27.792609 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:47:27.792609 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:47:27.792609 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:47:27.792609 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:47:27.792609 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:47:27.792609 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:47:27.792609 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:47:27.792609 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 15:47:27.792609 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 15:47:27.792609 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 15:47:27.792609 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 15:47:27.794289 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 15:47:27.794289 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 15:47:27.794289 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 15:47:27.794289 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 15:47:27.794289 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:47:27.794289 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:47:27.794289 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:47:27.794289 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:47:27.794289 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:47:27.794289 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:47:27.794289 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:47:27.794289 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Jun 23 15:47:27.794289 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:47:27.794289 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:47:27.794289 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:47:27.794289 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:47:27.796013 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:47:27.796013 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:47:27.796013 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:47:27.796013 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:47:27.796013 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 15:47:27.796013 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 15:47:27.796013 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 15:47:27.796013 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 15:47:27.796013 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 15:47:27.796013 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 15:47:27.796013 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 23 15:47:27.796013 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Jun 23 15:47:27.796013 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 163/337] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Jun 23 15:47:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:47:27.802952 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd8d0af-d6a5-4cd5-9bab-535e45b9216d None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:28.070929 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd8d0af-d6a5-4cd5-9bab-535e45b9216d None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:28.124502 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7a2fd1-ccac-45e8-9ec7-fc358b48a07e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:28.125249 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7a2fd1-ccac-45e8-9ec7-fc358b48a07e None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:28.125539 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7a2fd1-ccac-45e8-9ec7-fc358b48a07e None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:28.125814 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7a2fd1-ccac-45e8-9ec7-fc358b48a07e None admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:28.126581 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7a2fd1-ccac-45e8-9ec7-fc358b48a07e None admin] RBAC: Adding query filter params (name=member) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:47:28.126902 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7a2fd1-ccac-45e8-9ec7-fc358b48a07e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:28.128366 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7a2fd1-ccac-45e8-9ec7-fc358b48a07e None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:28.143013 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 175/338] 10.222.0.46 () {56 vars in 1101 bytes} [Thu Jun 23 15:47:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 340 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:47:28.148693 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3714734-3bf3-43e0-ae2d-971d6700655b None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:28.296861 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:28.300833 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3714734-3bf3-43e0-ae2d-971d6700655b None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:28.345828 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7e513a-5bc0-467f-9b70-913aab510b8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:28.346600 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7e513a-5bc0-467f-9b70-913aab510b8d None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:28.346806 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7e513a-5bc0-467f-9b70-913aab510b8d None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:28.347082 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7e513a-5bc0-467f-9b70-913aab510b8d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:28.348253 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5c1e83-c42e-4d73-b8a0-7f0c221fb3d4 None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:28.348545 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5c1e83-c42e-4d73-b8a0-7f0c221fb3d4 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:28.348818 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5c1e83-c42e-4d73-b8a0-7f0c221fb3d4 None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:28.350319 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 176/339] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jun 23 15:47:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 15:47:28.364308 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e687f1-b1a0-470a-97bb-4cb13a555a9d None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:28.364610 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e687f1-b1a0-470a-97bb-4cb13a555a9d None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:28.364858 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e687f1-b1a0-470a-97bb-4cb13a555a9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:28.399745 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-e5e687f1-b1a0-470a-97bb-4cb13a555a9d None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:47:28.401233 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-e5e687f1-b1a0-470a-97bb-4cb13a555a9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:47:28.451306 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:28.458511 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7e513a-5bc0-467f-9b70-913aab510b8d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:28.460026 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7e513a-5bc0-467f-9b70-913aab510b8d None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:28.549446 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 177/340] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jun 23 15:47:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 15:47:28.589372 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e687f1-b1a0-470a-97bb-4cb13a555a9d None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:28.634111 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:28.638265 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 164/341] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Jun 23 15:47:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 490 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 15:47:28.644120 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7e513a-5bc0-467f-9b70-913aab510b8d None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:28.765864 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7e513a-5bc0-467f-9b70-913aab510b8d None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:28.802906 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae668d6-1997-4639-818b-044765831734 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:28.803784 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae668d6-1997-4639-818b-044765831734 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:28.804062 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae668d6-1997-4639-818b-044765831734 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:28.804869 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae668d6-1997-4639-818b-044765831734 None admin] PATH_INFO: `/v3/projects/27f259ec3bd04186a70598926b09aa56` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:28.806866 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae668d6-1997-4639-818b-044765831734 None admin] RBAC: Authorizing `identity:get_project(project_id=27f259ec3bd04186a70598926b09aa56)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:28.807933 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e687f1-b1a0-470a-97bb-4cb13a555a9d None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:28.808311 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae668d6-1997-4639-818b-044765831734 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:28.810352 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 165/342] 10.222.0.46 () {56 vars in 1186 bytes} [Thu Jun 23 15:47:28 2022] GET /identity/v3/projects/27f259ec3bd04186a70598926b09aa56 => generated 298 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 15:47:28.815715 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae668d6-1997-4639-818b-044765831734 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:28.856525 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099fecb1-3bfe-41d1-8a4f-a298e4869ed2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:28.857414 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099fecb1-3bfe-41d1-8a4f-a298e4869ed2 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:28.857702 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099fecb1-3bfe-41d1-8a4f-a298e4869ed2 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:28.857973 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099fecb1-3bfe-41d1-8a4f-a298e4869ed2 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:28.876020 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099fecb1-3bfe-41d1-8a4f-a298e4869ed2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 15:47:28.888370 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-099fecb1-3bfe-41d1-8a4f-a298e4869ed2 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 15:47:28.888370 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:47:28.888370 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:47:28.888370 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:47:28.888370 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:47:28.888370 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:47:28.888370 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:47:28.888370 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:47:28.888370 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:47:28.888370 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:47:28.888370 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:47:28.888370 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:47:28.888370 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 15:47:28.888370 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 15:47:28.888370 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 15:47:28.888370 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 15:47:28.889283 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 15:47:28.889283 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 15:47:28.889283 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 15:47:28.889283 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 15:47:28.889283 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:47:28.889283 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:47:28.889283 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:47:28.889283 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:47:28.889283 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:47:28.889283 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:47:28.889283 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:47:28.889283 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Jun 23 15:47:28.889283 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:47:28.889283 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:47:28.889283 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:47:28.889283 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:47:28.890056 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:47:28.890056 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:47:28.890056 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:47:28.890056 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:47:28.890056 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 15:47:28.890056 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 15:47:28.890056 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 15:47:28.890056 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 15:47:28.890056 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 15:47:28.890056 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 15:47:28.890056 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 15:47:28.890056 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Jun 23 15:47:28.890056 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 178/343] 10.222.0.46 () {56 vars in 1114 bytes} [Thu Jun 23 15:47:28 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:47:28.891241 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f92c00-9d2c-4b68-b387-fda25afef239 None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:28.891524 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f92c00-9d2c-4b68-b387-fda25afef239 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:28.891798 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f92c00-9d2c-4b68-b387-fda25afef239 None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:28.893408 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 179/344] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jun 23 15:47:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 15:47:28.898531 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf2c5ff-fcf6-40ba-bc53-263e23cab817 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:28.898923 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf2c5ff-fcf6-40ba-bc53-263e23cab817 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:28.899201 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf2c5ff-fcf6-40ba-bc53-263e23cab817 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:28.918236 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-ddf2c5ff-fcf6-40ba-bc53-263e23cab817 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:47:28.919440 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-ddf2c5ff-fcf6-40ba-bc53-263e23cab817 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:47:28.938067 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:28.939078 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:28.989003 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae668d6-1997-4639-818b-044765831734 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:29.001416 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 180/345] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jun 23 15:47:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 15:47:29.002360 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf2c5ff-fcf6-40ba-bc53-263e23cab817 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:29.028706 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1fbf19-919b-441b-b480-b99e8ad9f191 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:29.029575 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1fbf19-919b-441b-b480-b99e8ad9f191 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:29.029855 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1fbf19-919b-441b-b480-b99e8ad9f191 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:29.030128 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1fbf19-919b-441b-b480-b99e8ad9f191 None admin] PATH_INFO: `/v3/groups/556c754e31a74b81bc55c99683c91208` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:29.043435 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1fbf19-919b-441b-b480-b99e8ad9f191 None admin] RBAC: Authorizing `identity:get_group(group_id=556c754e31a74b81bc55c99683c91208)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:29.044906 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1fbf19-919b-441b-b480-b99e8ad9f191 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:29.046951 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 166/346] 10.222.0.46 () {56 vars in 1180 bytes} [Thu Jun 23 15:47:28 2022] GET /identity/v3/groups/556c754e31a74b81bc55c99683c91208 => generated 226 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 15:47:29.047791 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1fbf19-919b-441b-b480-b99e8ad9f191 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:29.120087 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf2c5ff-fcf6-40ba-bc53-263e23cab817 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:29.164611 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fada7ea6-1e05-4909-be1e-305e74747e63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:29.165209 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fada7ea6-1e05-4909-be1e-305e74747e63 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:29.165490 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fada7ea6-1e05-4909-be1e-305e74747e63 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:29.165770 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fada7ea6-1e05-4909-be1e-305e74747e63 None admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:29.166527 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fada7ea6-1e05-4909-be1e-305e74747e63 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:47:29.166854 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fada7ea6-1e05-4909-be1e-305e74747e63 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:29.168202 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1fbf19-919b-441b-b480-b99e8ad9f191 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:29.169782 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fada7ea6-1e05-4909-be1e-305e74747e63 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:29.179080 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 181/347] 10.222.0.46 () {56 vars in 1111 bytes} [Thu Jun 23 15:47:29 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:47:29.183795 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fada7ea6-1e05-4909-be1e-305e74747e63 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:29.198201 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:29.203629 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fc5acd-c958-4aae-a703-071faaf9adaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:29.204078 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:29.205359 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fc5acd-c958-4aae-a703-071faaf9adaf None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:29.205797 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fc5acd-c958-4aae-a703-071faaf9adaf None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:29.206221 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fc5acd-c958-4aae-a703-071faaf9adaf None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:29.222801 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fc5acd-c958-4aae-a703-071faaf9adaf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 15:47:29.231417 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-21fc5acd-c958-4aae-a703-071faaf9adaf None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 23 15:47:29.231417 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:47:29.231417 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:47:29.231417 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:47:29.231417 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:47:29.231417 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:47:29.231417 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:47:29.231417 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:47:29.231417 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:47:29.231417 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:47:29.231417 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:47:29.231417 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:47:29.231417 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 15:47:29.231417 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 15:47:29.231417 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 15:47:29.231417 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 15:47:29.232630 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 15:47:29.232630 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 15:47:29.232630 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 15:47:29.232630 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 15:47:29.232630 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:47:29.232630 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:47:29.232630 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:47:29.232630 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:47:29.232630 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:47:29.232630 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:47:29.232630 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:47:29.232630 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Jun 23 15:47:29.232630 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:47:29.232630 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:47:29.232630 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:47:29.232630 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:47:29.233602 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:47:29.233602 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:47:29.233602 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:47:29.233602 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:47:29.233602 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 15:47:29.233602 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 15:47:29.233602 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 15:47:29.233602 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 15:47:29.233602 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 15:47:29.233602 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 15:47:29.233602 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 23 15:47:29.233602 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Jun 23 15:47:29.235634 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 167/348] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Jun 23 15:47:29 2022] GET /identity/v3/roles/reader => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:47:29.237021 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bffbbbd-daf0-4581-911b-a58693c67a91 None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:29.237449 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bffbbbd-daf0-4581-911b-a58693c67a91 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:29.237852 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bffbbbd-daf0-4581-911b-a58693c67a91 None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:29.239276 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 168/349] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jun 23 15:47:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 15:47:29.240106 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bffbbbd-daf0-4581-911b-a58693c67a91 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:29.342443 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fada7ea6-1e05-4909-be1e-305e74747e63 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:29.360312 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bffbbbd-daf0-4581-911b-a58693c67a91 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:29.392781 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5f8e23-7546-4948-92f5-507702d080bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:29.394022 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5f8e23-7546-4948-92f5-507702d080bf None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:29.394022 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5f8e23-7546-4948-92f5-507702d080bf None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:29.394291 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5f8e23-7546-4948-92f5-507702d080bf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:29.395153 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5f8e23-7546-4948-92f5-507702d080bf None admin] RBAC: Adding query filter params (group.id=556c754e31a74b81bc55c99683c91208, role.id=e2a5bbf3200a4d4091d72bc7ff78c34d, scope.project.id=27f259ec3bd04186a70598926b09aa56) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:47:29.395886 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5f8e23-7546-4948-92f5-507702d080bf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:29.397804 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5f8e23-7546-4948-92f5-507702d080bf None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:29.404644 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c858b0-b62c-4561-8275-0ea4b718ac56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:29.405535 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 182/350] 10.222.0.46 () {56 vars in 1376 bytes} [Thu Jun 23 15:47:29 2022] GET /identity/v3/role_assignments?group.id=556c754e31a74b81bc55c99683c91208&scope.project.id=27f259ec3bd04186a70598926b09aa56&role.id=e2a5bbf3200a4d4091d72bc7ff78c34d => generated 260 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:47:29.405874 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c858b0-b62c-4561-8275-0ea4b718ac56 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:29.406362 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5f8e23-7546-4948-92f5-507702d080bf None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:29.406465 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c858b0-b62c-4561-8275-0ea4b718ac56 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:29.407018 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c858b0-b62c-4561-8275-0ea4b718ac56 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:29.514854 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:29.543758 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5f8e23-7546-4948-92f5-507702d080bf None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:29.549619 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:29.588160 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9d908d-6e68-4dbc-b92c-fe2a45c822fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:29.589387 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9d908d-6e68-4dbc-b92c-fe2a45c822fc None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:29.589898 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9d908d-6e68-4dbc-b92c-fe2a45c822fc None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:29.590019 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c858b0-b62c-4561-8275-0ea4b718ac56 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:29.590496 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9d908d-6e68-4dbc-b92c-fe2a45c822fc None admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:29.591505 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9d908d-6e68-4dbc-b92c-fe2a45c822fc None admin] RBAC: Adding query filter params (name=reader) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:47:29.591610 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c858b0-b62c-4561-8275-0ea4b718ac56 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:29.592120 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9d908d-6e68-4dbc-b92c-fe2a45c822fc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:29.594185 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9d908d-6e68-4dbc-b92c-fe2a45c822fc None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:29.605636 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 183/351] 10.222.0.46 () {56 vars in 1101 bytes} [Thu Jun 23 15:47:29 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:47:29.607357 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6126d4aa-742f-4bb3-ad85-70ea10f5b054 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:29.608440 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6126d4aa-742f-4bb3-ad85-70ea10f5b054 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:29.608952 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6126d4aa-742f-4bb3-ad85-70ea10f5b054 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:29.628935 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-6126d4aa-742f-4bb3-ad85-70ea10f5b054 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:47:29.630492 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-6126d4aa-742f-4bb3-ad85-70ea10f5b054 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:47:29.730226 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 184/352] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jun 23 15:47:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 15:47:29.731379 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6126d4aa-742f-4bb3-ad85-70ea10f5b054 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:29.757670 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 169/353] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Jun 23 15:47:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 518 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 15:47:29.764208 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96845301-92af-40e4-9ce1-fc382d0d8074 None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:29.764501 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96845301-92af-40e4-9ce1-fc382d0d8074 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:29.764780 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96845301-92af-40e4-9ce1-fc382d0d8074 None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:29.766319 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 170/354] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jun 23 15:47:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 15:47:29.766871 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96845301-92af-40e4-9ce1-fc382d0d8074 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:29.839944 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:29.890442 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6126d4aa-742f-4bb3-ad85-70ea10f5b054 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:29.905959 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:29.917477 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96845301-92af-40e4-9ce1-fc382d0d8074 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:29.930521 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f8226b-6dfd-4b99-9724-03d70feed4d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:29.931457 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f8226b-6dfd-4b99-9724-03d70feed4d5 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:29.931744 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f8226b-6dfd-4b99-9724-03d70feed4d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:29.932021 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f8226b-6dfd-4b99-9724-03d70feed4d5 None admin] PATH_INFO: `/v3/users/509c2a6e8deb48fcbcacd1f6319d6f8b` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:29.935568 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f8226b-6dfd-4b99-9724-03d70feed4d5 None admin] RBAC: Authorizing `identity:get_user(user_id=509c2a6e8deb48fcbcacd1f6319d6f8b)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:29.935568 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f8226b-6dfd-4b99-9724-03d70feed4d5 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:29.936988 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 185/355] 10.222.0.46 () {56 vars in 1177 bytes} [Thu Jun 23 15:47:29 2022] GET /identity/v3/users/509c2a6e8deb48fcbcacd1f6319d6f8b => generated 293 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:47:29.937722 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f8226b-6dfd-4b99-9724-03d70feed4d5 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:29.964027 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2741eec1-e951-43cc-8bde-1ba1c41fc907 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:29.966509 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2741eec1-e951-43cc-8bde-1ba1c41fc907 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:29.967060 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2741eec1-e951-43cc-8bde-1ba1c41fc907 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:29.967561 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2741eec1-e951-43cc-8bde-1ba1c41fc907 None admin] PATH_INFO: `/v3/projects/27f259ec3bd04186a70598926b09aa56` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:29.968845 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2741eec1-e951-43cc-8bde-1ba1c41fc907 None admin] RBAC: Authorizing `identity:get_project(project_id=27f259ec3bd04186a70598926b09aa56)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:29.970306 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2741eec1-e951-43cc-8bde-1ba1c41fc907 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:29.972236 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 171/356] 10.222.0.46 () {56 vars in 1186 bytes} [Thu Jun 23 15:47:29 2022] GET /identity/v3/projects/27f259ec3bd04186a70598926b09aa56 => generated 298 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:47:29.973345 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fac1d4d-925e-40cf-8298-b782b0a9bcd2 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:29.973542 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fac1d4d-925e-40cf-8298-b782b0a9bcd2 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:29.973822 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fac1d4d-925e-40cf-8298-b782b0a9bcd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:29.992148 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-4fac1d4d-925e-40cf-8298-b782b0a9bcd2 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:47:29.993477 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-4fac1d4d-925e-40cf-8298-b782b0a9bcd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:47:30.058270 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f8226b-6dfd-4b99-9724-03d70feed4d5 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:30.071790 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 172/357] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jun 23 15:47:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 15:47:30.072432 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fac1d4d-925e-40cf-8298-b782b0a9bcd2 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:30.107780 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c269062e-0449-488c-b665-5cc1d3e2ca97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:30.108592 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c269062e-0449-488c-b665-5cc1d3e2ca97 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:30.109035 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c269062e-0449-488c-b665-5cc1d3e2ca97 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:30.109324 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c269062e-0449-488c-b665-5cc1d3e2ca97 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:30.118813 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:30.123783 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c269062e-0449-488c-b665-5cc1d3e2ca97 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 15:47:30.133640 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-c269062e-0449-488c-b665-5cc1d3e2ca97 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 23 15:47:30.133640 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:47:30.133640 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:47:30.133640 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:47:30.133640 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:47:30.133640 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:47:30.133640 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:47:30.133640 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:47:30.133640 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:47:30.133640 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:47:30.133640 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:47:30.133640 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:47:30.133640 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 15:47:30.133640 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 15:47:30.133640 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 15:47:30.133640 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 15:47:30.134525 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 15:47:30.134525 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 15:47:30.134525 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 15:47:30.134525 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 15:47:30.134525 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:47:30.134525 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:47:30.134525 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:47:30.134525 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:47:30.134525 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:47:30.134525 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:47:30.134525 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:47:30.134525 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Jun 23 15:47:30.134525 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:47:30.134525 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:47:30.134525 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:47:30.134525 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:47:30.135392 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:47:30.135392 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:47:30.135392 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:47:30.135392 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:47:30.135392 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 15:47:30.135392 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 15:47:30.135392 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 15:47:30.135392 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 15:47:30.135392 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 15:47:30.135392 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 15:47:30.135392 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 23 15:47:30.135392 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Jun 23 15:47:30.136688 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 186/358] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Jun 23 15:47:29 2022] GET /identity/v3/roles/member => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:47:30.137021 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c269062e-0449-488c-b665-5cc1d3e2ca97 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:30.180891 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:30.193623 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fac1d4d-925e-40cf-8298-b782b0a9bcd2 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:30.239308 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b25f7fc-ae17-4d42-b2a3-b0a697de5209 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:30.240356 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b25f7fc-ae17-4d42-b2a3-b0a697de5209 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:30.240847 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b25f7fc-ae17-4d42-b2a3-b0a697de5209 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:30.242906 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b25f7fc-ae17-4d42-b2a3-b0a697de5209 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:30.243965 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b25f7fc-ae17-4d42-b2a3-b0a697de5209 None admin] RBAC: Adding query filter params (role.id=38bbc1cebab84cac9b213ffba44bb4a6, user.id=509c2a6e8deb48fcbcacd1f6319d6f8b, scope.system=all) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:47:30.244490 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b25f7fc-ae17-4d42-b2a3-b0a697de5209 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:30.246129 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b25f7fc-ae17-4d42-b2a3-b0a697de5209 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:30.260486 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 173/359] 10.222.0.46 () {56 vars in 1308 bytes} [Thu Jun 23 15:47:30 2022] GET /identity/v3/role_assignments?user.id=509c2a6e8deb48fcbcacd1f6319d6f8b&scope.system=all&role.id=38bbc1cebab84cac9b213ffba44bb4a6 => generated 510 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:47:30.266716 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aabced8-b662-4c38-b712-2a63c0c30d1e None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:30.266716 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aabced8-b662-4c38-b712-2a63c0c30d1e None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:30.266716 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aabced8-b662-4c38-b712-2a63c0c30d1e None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:30.266716 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 174/360] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jun 23 15:47:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 15:47:30.266716 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aabced8-b662-4c38-b712-2a63c0c30d1e None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:30.276214 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c269062e-0449-488c-b665-5cc1d3e2ca97 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:30.322912 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45efdd1e-0a03-41b7-b328-b421c50599f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:30.323714 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45efdd1e-0a03-41b7-b328-b421c50599f5 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:30.323997 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45efdd1e-0a03-41b7-b328-b421c50599f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:30.324278 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45efdd1e-0a03-41b7-b328-b421c50599f5 None admin] PATH_INFO: `/v3/groups/556c754e31a74b81bc55c99683c91208` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:30.325500 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45efdd1e-0a03-41b7-b328-b421c50599f5 None admin] RBAC: Authorizing `identity:get_group(group_id=556c754e31a74b81bc55c99683c91208)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:30.327093 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45efdd1e-0a03-41b7-b328-b421c50599f5 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:30.328959 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 187/361] 10.222.0.46 () {56 vars in 1180 bytes} [Thu Jun 23 15:47:30 2022] GET /identity/v3/groups/556c754e31a74b81bc55c99683c91208 => generated 226 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:47:30.332443 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45efdd1e-0a03-41b7-b328-b421c50599f5 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:30.413722 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aabced8-b662-4c38-b712-2a63c0c30d1e None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:30.459164 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:30.467004 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34a082b-8463-4e04-bd63-22b9cccc60f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:30.467004 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34a082b-8463-4e04-bd63-22b9cccc60f3 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:30.467486 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34a082b-8463-4e04-bd63-22b9cccc60f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:30.467779 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34a082b-8463-4e04-bd63-22b9cccc60f3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:30.486803 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45efdd1e-0a03-41b7-b328-b421c50599f5 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:30.487635 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34a082b-8463-4e04-bd63-22b9cccc60f3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 15:47:30.496448 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-a34a082b-8463-4e04-bd63-22b9cccc60f3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 15:47:30.496448 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:47:30.496448 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:47:30.496448 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:47:30.496448 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:47:30.496448 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:47:30.496448 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:47:30.496448 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:47:30.496448 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:47:30.496448 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:47:30.496448 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:47:30.496448 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:47:30.496448 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 15:47:30.496448 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 15:47:30.496448 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 15:47:30.496448 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 15:47:30.497390 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 15:47:30.497390 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 15:47:30.497390 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 15:47:30.497390 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 15:47:30.497390 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:47:30.497390 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:47:30.497390 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:47:30.497390 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:47:30.497390 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:47:30.497390 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:47:30.497390 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:47:30.497390 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Jun 23 15:47:30.497390 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:47:30.497390 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:47:30.497390 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:47:30.497390 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:47:30.498229 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:47:30.498229 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:47:30.498229 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:47:30.498229 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:47:30.498229 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 15:47:30.498229 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 15:47:30.498229 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 15:47:30.498229 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 15:47:30.498229 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 15:47:30.498229 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 15:47:30.498229 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 15:47:30.498229 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Jun 23 15:47:30.498229 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 175/362] 10.222.0.46 () {56 vars in 1114 bytes} [Thu Jun 23 15:47:30 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:47:30.499680 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a1419e-6d60-44fc-a64e-f07dea20c83c None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:30.499970 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a1419e-6d60-44fc-a64e-f07dea20c83c None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:30.500246 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a1419e-6d60-44fc-a64e-f07dea20c83c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:30.520926 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-f2a1419e-6d60-44fc-a64e-f07dea20c83c None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:47:30.522152 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-f2a1419e-6d60-44fc-a64e-f07dea20c83c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:47:30.524575 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:30.543806 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7186ffda-4646-4450-aedb-44f4e34a1b07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:30.544493 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7186ffda-4646-4450-aedb-44f4e34a1b07 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:30.544782 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7186ffda-4646-4450-aedb-44f4e34a1b07 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:30.545067 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7186ffda-4646-4450-aedb-44f4e34a1b07 None admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:30.546127 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7186ffda-4646-4450-aedb-44f4e34a1b07 None admin] RBAC: Adding query filter params (name=member) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:47:30.546127 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7186ffda-4646-4450-aedb-44f4e34a1b07 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:30.547584 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7186ffda-4646-4450-aedb-44f4e34a1b07 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:30.564703 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 188/363] 10.222.0.46 () {56 vars in 1101 bytes} [Thu Jun 23 15:47:30 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:47:30.565750 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7186ffda-4646-4450-aedb-44f4e34a1b07 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:30.609714 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 176/364] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jun 23 15:47:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 15:47:30.615679 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a1419e-6d60-44fc-a64e-f07dea20c83c None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:30.711192 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7186ffda-4646-4450-aedb-44f4e34a1b07 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:30.748825 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:30.752084 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a1419e-6d60-44fc-a64e-f07dea20c83c None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:30.760744 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7df4972-15ee-44de-a5d9-78cc1e730ce3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:30.761413 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7df4972-15ee-44de-a5d9-78cc1e730ce3 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:30.761699 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7df4972-15ee-44de-a5d9-78cc1e730ce3 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:30.761976 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7df4972-15ee-44de-a5d9-78cc1e730ce3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:30.763205 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7df4972-15ee-44de-a5d9-78cc1e730ce3 None admin] RBAC: Adding query filter params (group.id=556c754e31a74b81bc55c99683c91208, role.id=d4d557a3ec4e434f8e83499c5fcc7136, scope.project.id=27f259ec3bd04186a70598926b09aa56) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:47:30.763205 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7df4972-15ee-44de-a5d9-78cc1e730ce3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:30.765217 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7df4972-15ee-44de-a5d9-78cc1e730ce3 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:30.771619 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 189/365] 10.222.0.46 () {56 vars in 1376 bytes} [Thu Jun 23 15:47:30 2022] GET /identity/v3/role_assignments?group.id=556c754e31a74b81bc55c99683c91208&scope.project.id=27f259ec3bd04186a70598926b09aa56&role.id=d4d557a3ec4e434f8e83499c5fcc7136 => generated 260 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:47:30.772774 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7df4972-15ee-44de-a5d9-78cc1e730ce3 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:30.803287 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f456ee-c5cd-4aee-a53d-d0345b615761 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:30.805621 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f456ee-c5cd-4aee-a53d-d0345b615761 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:30.806481 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f456ee-c5cd-4aee-a53d-d0345b615761 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:30.806997 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f456ee-c5cd-4aee-a53d-d0345b615761 None admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:30.808172 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f456ee-c5cd-4aee-a53d-d0345b615761 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:47:30.809046 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f456ee-c5cd-4aee-a53d-d0345b615761 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:30.811197 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f456ee-c5cd-4aee-a53d-d0345b615761 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:30.818971 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:30.826873 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 177/366] 10.222.0.46 () {56 vars in 1111 bytes} [Thu Jun 23 15:47:30 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:47:30.828118 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f456ee-c5cd-4aee-a53d-d0345b615761 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:30.926592 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7df4972-15ee-44de-a5d9-78cc1e730ce3 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:30.971408 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f456ee-c5cd-4aee-a53d-d0345b615761 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:30.975195 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcaeadf-14d4-4a15-a3ef-9e4437cc7b12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:30.977379 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcaeadf-14d4-4a15-a3ef-9e4437cc7b12 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:30.977681 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcaeadf-14d4-4a15-a3ef-9e4437cc7b12 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:30.977968 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcaeadf-14d4-4a15-a3ef-9e4437cc7b12 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:31.018643 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2887c0ce-3311-439c-830b-874bb7f59e3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:31.019605 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2887c0ce-3311-439c-830b-874bb7f59e3e None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:31.019891 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2887c0ce-3311-439c-830b-874bb7f59e3e None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:31.020172 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2887c0ce-3311-439c-830b-874bb7f59e3e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:31.032629 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2887c0ce-3311-439c-830b-874bb7f59e3e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 15:47:31.041667 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-2887c0ce-3311-439c-830b-874bb7f59e3e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 15:47:31.041667 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:47:31.041667 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:47:31.041667 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:47:31.041667 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:47:31.041667 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:47:31.041667 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:47:31.041667 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:47:31.041667 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:47:31.041667 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:47:31.041667 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:47:31.041667 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:47:31.041667 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 15:47:31.041667 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 15:47:31.041667 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 15:47:31.041667 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 15:47:31.042559 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 15:47:31.042559 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 15:47:31.042559 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 15:47:31.042559 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 15:47:31.042559 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:47:31.042559 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:47:31.042559 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:47:31.042559 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:47:31.042559 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:47:31.042559 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:47:31.042559 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:47:31.042559 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Jun 23 15:47:31.042559 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:47:31.042559 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:47:31.042559 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:47:31.042559 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:47:31.043394 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:47:31.043394 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:47:31.043394 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:47:31.043394 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:47:31.043394 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 15:47:31.043394 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 15:47:31.043394 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 15:47:31.043394 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 15:47:31.043394 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 15:47:31.043394 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 15:47:31.043394 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 15:47:31.043394 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Jun 23 15:47:31.044032 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 178/367] 10.222.0.46 () {56 vars in 1096 bytes} [Thu Jun 23 15:47:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:47:31.044601 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2887c0ce-3311-439c-830b-874bb7f59e3e None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:31.058034 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:31.072215 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bcaeadf-14d4-4a15-a3ef-9e4437cc7b12 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:31.074028 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bcaeadf-14d4-4a15-a3ef-9e4437cc7b12 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:31.115516 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:31.165711 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2887c0ce-3311-439c-830b-874bb7f59e3e None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:31.211544 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 190/368] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Jun 23 15:47:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 15:47:31.211544 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcaeadf-14d4-4a15-a3ef-9e4437cc7b12 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:31.212521 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6618fde-748d-4604-b9ae-d03d83a80cd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:31.213440 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6618fde-748d-4604-b9ae-d03d83a80cd6 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:31.213919 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6618fde-748d-4604-b9ae-d03d83a80cd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:31.214387 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6618fde-748d-4604-b9ae-d03d83a80cd6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:31.304444 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6618fde-748d-4604-b9ae-d03d83a80cd6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:31.305907 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6618fde-748d-4604-b9ae-d03d83a80cd6 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:31.321125 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:31.335974 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcaeadf-14d4-4a15-a3ef-9e4437cc7b12 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:31.376132 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:31.386550 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb32f006-de1d-40d8-99db-5b10ece74bfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:31.387294 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb32f006-de1d-40d8-99db-5b10ece74bfc None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:31.387588 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb32f006-de1d-40d8-99db-5b10ece74bfc None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:31.387871 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb32f006-de1d-40d8-99db-5b10ece74bfc None admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:31.389294 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb32f006-de1d-40d8-99db-5b10ece74bfc None admin] RBAC: Adding query filter params (name=admin) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:47:31.389617 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb32f006-de1d-40d8-99db-5b10ece74bfc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:31.391774 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb32f006-de1d-40d8-99db-5b10ece74bfc None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:31.404370 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 191/369] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Jun 23 15:47:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:47:31.405250 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb32f006-de1d-40d8-99db-5b10ece74bfc None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:31.432873 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 179/370] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Jun 23 15:47:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 15:47:31.433741 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6618fde-748d-4604-b9ae-d03d83a80cd6 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:31.532509 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb32f006-de1d-40d8-99db-5b10ece74bfc None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:31.573720 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02f3695-5fe5-4fa7-9cb3-ae41cd4c85ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:31.573720 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02f3695-5fe5-4fa7-9cb3-ae41cd4c85ba None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:31.573720 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02f3695-5fe5-4fa7-9cb3-ae41cd4c85ba None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:31.573720 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02f3695-5fe5-4fa7-9cb3-ae41cd4c85ba None admin] PATH_INFO: `/v3/projects/2240bbdc36ed4e59a7454b287b5891ab` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:31.576576 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02f3695-5fe5-4fa7-9cb3-ae41cd4c85ba None admin] RBAC: Authorizing `identity:get_project(project_id=2240bbdc36ed4e59a7454b287b5891ab)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:31.578135 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02f3695-5fe5-4fa7-9cb3-ae41cd4c85ba None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:31.580242 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 192/371] 10.222.0.46 () {56 vars in 1186 bytes} [Thu Jun 23 15:47:31 2022] GET /identity/v3/projects/2240bbdc36ed4e59a7454b287b5891ab => generated 302 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:47:31.581092 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02f3695-5fe5-4fa7-9cb3-ae41cd4c85ba None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:31.583032 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6618fde-748d-4604-b9ae-d03d83a80cd6 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:31.614464 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:31.629752 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9243d0fe-ceed-4fce-9544-16217eaabfc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:31.631462 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9243d0fe-ceed-4fce-9544-16217eaabfc2 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:31.631750 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9243d0fe-ceed-4fce-9544-16217eaabfc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:31.632027 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9243d0fe-ceed-4fce-9544-16217eaabfc2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:31.677746 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:31.724423 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02f3695-5fe5-4fa7-9cb3-ae41cd4c85ba None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:31.729687 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9243d0fe-ceed-4fce-9544-16217eaabfc2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:31.731231 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9243d0fe-ceed-4fce-9544-16217eaabfc2 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:31.778619 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f420b8-3a98-4269-8af9-17e6df5df857 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:31.779656 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f420b8-3a98-4269-8af9-17e6df5df857 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:31.780017 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f420b8-3a98-4269-8af9-17e6df5df857 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:31.780368 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f420b8-3a98-4269-8af9-17e6df5df857 None admin] PATH_INFO: `/v3/projects/2240bbdc36ed4e59a7454b287b5891ab` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:31.781674 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f420b8-3a98-4269-8af9-17e6df5df857 None admin] RBAC: Authorizing `identity:get_project(project_id=2240bbdc36ed4e59a7454b287b5891ab)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:31.783272 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f420b8-3a98-4269-8af9-17e6df5df857 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:31.785264 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 193/372] 10.222.0.46 () {56 vars in 1186 bytes} [Thu Jun 23 15:47:31 2022] GET /identity/v3/projects/2240bbdc36ed4e59a7454b287b5891ab => generated 302 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:47:31.786124 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f420b8-3a98-4269-8af9-17e6df5df857 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:31.860293 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 180/373] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Jun 23 15:47:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 15:47:31.861596 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9243d0fe-ceed-4fce-9544-16217eaabfc2 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:31.911751 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:31.928419 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f420b8-3a98-4269-8af9-17e6df5df857 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:31.933133 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:31.976828 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cdeb5a-d571-43b0-b95c-581876b45736 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:31.978140 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cdeb5a-d571-43b0-b95c-581876b45736 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:31.978588 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cdeb5a-d571-43b0-b95c-581876b45736 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:31.978948 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cdeb5a-d571-43b0-b95c-581876b45736 None admin] PATH_INFO: `/v3/groups/556c754e31a74b81bc55c99683c91208` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:31.980169 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8cdeb5a-d571-43b0-b95c-581876b45736 None admin] RBAC: Authorizing `identity:get_group(group_id=556c754e31a74b81bc55c99683c91208)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:31.981637 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8cdeb5a-d571-43b0-b95c-581876b45736 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:31.983528 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 194/374] 10.222.0.46 () {56 vars in 1180 bytes} [Thu Jun 23 15:47:31 2022] GET /identity/v3/groups/556c754e31a74b81bc55c99683c91208 => generated 226 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:47:31.984585 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff099fd3-a82b-4c1d-bd8e-8a11ec6d2e1a None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:31.984874 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff099fd3-a82b-4c1d-bd8e-8a11ec6d2e1a None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:31.985156 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff099fd3-a82b-4c1d-bd8e-8a11ec6d2e1a None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:31.986567 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 195/375] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jun 23 15:47:31 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 15:47:31.987159 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff099fd3-a82b-4c1d-bd8e-8a11ec6d2e1a None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:31.995320 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9243d0fe-ceed-4fce-9544-16217eaabfc2 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:32.039067 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acad77d2-cbd9-45e4-99e8-7478af61b1f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:32.040472 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acad77d2-cbd9-45e4-99e8-7478af61b1f6 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:32.041089 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acad77d2-cbd9-45e4-99e8-7478af61b1f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:32.041599 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acad77d2-cbd9-45e4-99e8-7478af61b1f6 None admin] PATH_INFO: `/v3/groups/556c754e31a74b81bc55c99683c91208` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:32.043417 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acad77d2-cbd9-45e4-99e8-7478af61b1f6 None admin] RBAC: Authorizing `identity:get_group(group_id=556c754e31a74b81bc55c99683c91208)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:32.045221 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acad77d2-cbd9-45e4-99e8-7478af61b1f6 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:32.047597 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 181/376] 10.222.0.46 () {56 vars in 1180 bytes} [Thu Jun 23 15:47:31 2022] GET /identity/v3/groups/556c754e31a74b81bc55c99683c91208 => generated 226 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:47:32.054747 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06dd37d-0fd7-40d7-8448-0de6c4de54a5 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:32.054747 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06dd37d-0fd7-40d7-8448-0de6c4de54a5 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:32.054747 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06dd37d-0fd7-40d7-8448-0de6c4de54a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:32.070544 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-b06dd37d-0fd7-40d7-8448-0de6c4de54a5 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:47:32.072009 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-b06dd37d-0fd7-40d7-8448-0de6c4de54a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:47:32.130371 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff099fd3-a82b-4c1d-bd8e-8a11ec6d2e1a None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:32.169725 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 182/377] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jun 23 15:47:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 15:47:32.170901 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06dd37d-0fd7-40d7-8448-0de6c4de54a5 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:32.176161 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea68217b-6d59-44f4-b3bd-433cbbd563ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:32.177306 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea68217b-6d59-44f4-b3bd-433cbbd563ea None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:32.177802 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea68217b-6d59-44f4-b3bd-433cbbd563ea None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:32.178277 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea68217b-6d59-44f4-b3bd-433cbbd563ea None admin] PATH_INFO: `/v3/projects/110dee34db4740f989f715e757ee809c` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:32.180507 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea68217b-6d59-44f4-b3bd-433cbbd563ea None admin] RBAC: Authorizing `identity:get_project(project_id=110dee34db4740f989f715e757ee809c)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:32.182400 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea68217b-6d59-44f4-b3bd-433cbbd563ea None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:32.184586 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 196/378] 10.222.0.46 () {56 vars in 1186 bytes} [Thu Jun 23 15:47:31 2022] GET /identity/v3/projects/110dee34db4740f989f715e757ee809c => generated 344 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:47:32.187876 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea68217b-6d59-44f4-b3bd-433cbbd563ea None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:32.225375 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:32.247542 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:32.302113 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06dd37d-0fd7-40d7-8448-0de6c4de54a5 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:32.322427 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea68217b-6d59-44f4-b3bd-433cbbd563ea None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:32.348433 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814bcdff-f41b-4a4c-b840-5f9793e90b15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:32.350725 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814bcdff-f41b-4a4c-b840-5f9793e90b15 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:32.350725 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814bcdff-f41b-4a4c-b840-5f9793e90b15 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:32.350725 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814bcdff-f41b-4a4c-b840-5f9793e90b15 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:32.351981 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814bcdff-f41b-4a4c-b840-5f9793e90b15 None admin] RBAC: Adding query filter params (group.id=556c754e31a74b81bc55c99683c91208, role.id=e2a5bbf3200a4d4091d72bc7ff78c34d, scope.project.id=2240bbdc36ed4e59a7454b287b5891ab) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:47:32.352151 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814bcdff-f41b-4a4c-b840-5f9793e90b15 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:32.354328 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814bcdff-f41b-4a4c-b840-5f9793e90b15 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:32.361359 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 183/379] 10.222.0.46 () {56 vars in 1376 bytes} [Thu Jun 23 15:47:32 2022] GET /identity/v3/role_assignments?group.id=556c754e31a74b81bc55c99683c91208&scope.project.id=2240bbdc36ed4e59a7454b287b5891ab&role.id=e2a5bbf3200a4d4091d72bc7ff78c34d => generated 260 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:47:32.361988 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814bcdff-f41b-4a4c-b840-5f9793e90b15 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:32.368012 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5b990e-0346-458a-a316-4553ad645b52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:32.368696 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5b990e-0346-458a-a316-4553ad645b52 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:32.368979 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5b990e-0346-458a-a316-4553ad645b52 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:32.369252 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5b990e-0346-458a-a316-4553ad645b52 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:32.370098 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5b990e-0346-458a-a316-4553ad645b52 None admin] RBAC: Adding query filter params (group.id=556c754e31a74b81bc55c99683c91208, role.id=d4d557a3ec4e434f8e83499c5fcc7136, scope.project.id=2240bbdc36ed4e59a7454b287b5891ab) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:47:32.370409 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5b990e-0346-458a-a316-4553ad645b52 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:32.374370 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5b990e-0346-458a-a316-4553ad645b52 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:32.378718 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 197/380] 10.222.0.46 () {56 vars in 1376 bytes} [Thu Jun 23 15:47:32 2022] GET /identity/v3/role_assignments?group.id=556c754e31a74b81bc55c99683c91208&scope.project.id=2240bbdc36ed4e59a7454b287b5891ab&role.id=d4d557a3ec4e434f8e83499c5fcc7136 => generated 260 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:47:32.379454 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5b990e-0346-458a-a316-4553ad645b52 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:32.496319 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814bcdff-f41b-4a4c-b840-5f9793e90b15 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:32.544452 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb9ade4-eece-46ff-a0c7-22f2d73f8c53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:32.545550 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb9ade4-eece-46ff-a0c7-22f2d73f8c53 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:32.546894 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb9ade4-eece-46ff-a0c7-22f2d73f8c53 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:32.547405 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb9ade4-eece-46ff-a0c7-22f2d73f8c53 None admin] PATH_INFO: `/v3/groups/47834ccd8b0046b2920f4352334f4694` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:32.553592 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb9ade4-eece-46ff-a0c7-22f2d73f8c53 None admin] RBAC: Authorizing `identity:get_group(group_id=47834ccd8b0046b2920f4352334f4694)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:32.562095 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb9ade4-eece-46ff-a0c7-22f2d73f8c53 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:32.565054 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 184/381] 10.222.0.46 () {56 vars in 1180 bytes} [Thu Jun 23 15:47:32 2022] GET /identity/v3/groups/47834ccd8b0046b2920f4352334f4694 => generated 229 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:47:32.565479 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:32.574743 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb9ade4-eece-46ff-a0c7-22f2d73f8c53 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:32.671657 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:32.686983 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5b990e-0346-458a-a316-4553ad645b52 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:32.713881 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb9ade4-eece-46ff-a0c7-22f2d73f8c53 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:32.755565 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532fe329-2b21-402c-b425-cf8ec5446111 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:32.756802 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532fe329-2b21-402c-b425-cf8ec5446111 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:32.757283 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532fe329-2b21-402c-b425-cf8ec5446111 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:32.757598 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532fe329-2b21-402c-b425-cf8ec5446111 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:32.758479 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532fe329-2b21-402c-b425-cf8ec5446111 None admin] RBAC: Adding query filter params (group.id=47834ccd8b0046b2920f4352334f4694, role.id=812d3c567c52469cb2b35709fbdc857c, scope.project.id=110dee34db4740f989f715e757ee809c) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:47:32.758822 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532fe329-2b21-402c-b425-cf8ec5446111 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:32.760296 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532fe329-2b21-402c-b425-cf8ec5446111 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:32.767097 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 185/382] 10.222.0.46 () {56 vars in 1376 bytes} [Thu Jun 23 15:47:32 2022] GET /identity/v3/role_assignments?group.id=47834ccd8b0046b2920f4352334f4694&scope.project.id=110dee34db4740f989f715e757ee809c&role.id=812d3c567c52469cb2b35709fbdc857c => generated 260 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:47:32.775516 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae5fcf0-fef4-4672-a904-3f70252a0d49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:32.776643 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae5fcf0-fef4-4672-a904-3f70252a0d49 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:32.776936 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae5fcf0-fef4-4672-a904-3f70252a0d49 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:32.777224 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae5fcf0-fef4-4672-a904-3f70252a0d49 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:32.803989 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae5fcf0-fef4-4672-a904-3f70252a0d49 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 15:47:32.812088 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-7ae5fcf0-fef4-4672-a904-3f70252a0d49 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 23 15:47:32.812088 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:47:32.812088 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:47:32.812088 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:47:32.812088 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:47:32.812088 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:47:32.812088 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:47:32.812088 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:47:32.812088 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:47:32.812088 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:47:32.812088 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:47:32.812088 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:47:32.812088 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 15:47:32.812088 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 15:47:32.812088 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 15:47:32.812088 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 15:47:32.813010 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 15:47:32.813010 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 15:47:32.813010 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 15:47:32.813010 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 15:47:32.813010 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:47:32.813010 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:47:32.813010 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:47:32.813010 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:47:32.813010 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:47:32.813010 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:47:32.813010 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:47:32.813010 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Jun 23 15:47:32.813010 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:47:32.813010 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:47:32.813010 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:47:32.813010 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:47:32.813828 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:47:32.813828 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:47:32.813828 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:47:32.813828 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:47:32.813828 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 15:47:32.813828 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 15:47:32.813828 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 15:47:32.813828 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 15:47:32.813828 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 15:47:32.813828 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 15:47:32.813828 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 23 15:47:32.813828 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Jun 23 15:47:32.813828 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 198/383] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Jun 23 15:47:32 2022] GET /identity/v3/roles/member => generated 84 bytes in 434 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:47:32.821120 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532fe329-2b21-402c-b425-cf8ec5446111 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:32.882941 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:32.950086 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532fe329-2b21-402c-b425-cf8ec5446111 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:33.000889 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303ff69f-06fd-474e-bbac-73a3f4190086 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:33.001589 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303ff69f-06fd-474e-bbac-73a3f4190086 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:33.001874 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303ff69f-06fd-474e-bbac-73a3f4190086 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:33.002154 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303ff69f-06fd-474e-bbac-73a3f4190086 None admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:33.002946 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303ff69f-06fd-474e-bbac-73a3f4190086 None admin] RBAC: Adding query filter params (name=member) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:47:33.003259 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303ff69f-06fd-474e-bbac-73a3f4190086 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:33.004658 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303ff69f-06fd-474e-bbac-73a3f4190086 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:33.015104 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 186/384] 10.222.0.46 () {56 vars in 1101 bytes} [Thu Jun 23 15:47:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:47:33.020227 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae5fcf0-fef4-4672-a904-3f70252a0d49 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:33.157416 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337fc2a8-f14e-4bca-9c53-9f293dcdc3ee None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:33.157766 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337fc2a8-f14e-4bca-9c53-9f293dcdc3ee None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:33.157984 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337fc2a8-f14e-4bca-9c53-9f293dcdc3ee None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:33.159388 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 187/385] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jun 23 15:47:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 15:47:33.174649 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b03fd57-4931-4632-ad4c-331c7874fda4 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:33.174957 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b03fd57-4931-4632-ad4c-331c7874fda4 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:33.175244 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b03fd57-4931-4632-ad4c-331c7874fda4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:33.186052 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae5fcf0-fef4-4672-a904-3f70252a0d49 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:33.219740 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-1b03fd57-4931-4632-ad4c-331c7874fda4 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:47:33.220949 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-1b03fd57-4931-4632-ad4c-331c7874fda4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:47:33.225898 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcea2ad-0938-4a45-93bc-1ca519c76425 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:33.226797 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcea2ad-0938-4a45-93bc-1ca519c76425 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:33.227086 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcea2ad-0938-4a45-93bc-1ca519c76425 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:33.227356 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcea2ad-0938-4a45-93bc-1ca519c76425 None admin] PATH_INFO: `/v3/groups/556c754e31a74b81bc55c99683c91208` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:33.228547 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dcea2ad-0938-4a45-93bc-1ca519c76425 None admin] RBAC: Authorizing `identity:get_group(group_id=556c754e31a74b81bc55c99683c91208)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:33.230021 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dcea2ad-0938-4a45-93bc-1ca519c76425 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:33.231790 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 199/386] 10.222.0.46 () {56 vars in 1180 bytes} [Thu Jun 23 15:47:33 2022] GET /identity/v3/groups/556c754e31a74b81bc55c99683c91208 => generated 226 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:47:33.239730 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcea2ad-0938-4a45-93bc-1ca519c76425 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:33.262186 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:33.348441 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 188/387] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jun 23 15:47:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 15:47:33.388057 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b03fd57-4931-4632-ad4c-331c7874fda4 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:33.403471 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcea2ad-0938-4a45-93bc-1ca519c76425 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:33.470196 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:33.473623 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acefd1d6-3573-4e35-ba63-c14da18a80cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:33.476392 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acefd1d6-3573-4e35-ba63-c14da18a80cb None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:33.476952 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acefd1d6-3573-4e35-ba63-c14da18a80cb None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:33.478242 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acefd1d6-3573-4e35-ba63-c14da18a80cb None admin] PATH_INFO: `/v3/projects/27f259ec3bd04186a70598926b09aa56` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:33.480498 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acefd1d6-3573-4e35-ba63-c14da18a80cb None admin] RBAC: Authorizing `identity:get_project(project_id=27f259ec3bd04186a70598926b09aa56)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:33.482328 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acefd1d6-3573-4e35-ba63-c14da18a80cb None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:33.486836 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 200/388] 10.222.0.46 () {56 vars in 1186 bytes} [Thu Jun 23 15:47:33 2022] GET /identity/v3/projects/27f259ec3bd04186a70598926b09aa56 => generated 298 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:47:33.492066 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acefd1d6-3573-4e35-ba63-c14da18a80cb None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:33.605606 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b03fd57-4931-4632-ad4c-331c7874fda4 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:33.621230 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acefd1d6-3573-4e35-ba63-c14da18a80cb None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:33.646372 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:33.659677 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5dbc0d-9f57-46dd-93b0-8b45ff8d545a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:33.660778 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5dbc0d-9f57-46dd-93b0-8b45ff8d545a None admin] REQUEST_METHOD: `PUT` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:33.661200 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5dbc0d-9f57-46dd-93b0-8b45ff8d545a None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:33.662586 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5dbc0d-9f57-46dd-93b0-8b45ff8d545a None admin] PATH_INFO: `/v3/projects/27f259ec3bd04186a70598926b09aa56/groups/556c754e31a74b81bc55c99683c91208/roles/e2a5bbf3200a4d4091d72bc7ff78c34d` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:33.665270 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5dbc0d-9f57-46dd-93b0-8b45ff8d545a None admin] RBAC: Authorizing `identity:create_grant(project_id=27f259ec3bd04186a70598926b09aa56, group_id=556c754e31a74b81bc55c99683c91208, role_id=e2a5bbf3200a4d4091d72bc7ff78c34d)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:33.666882 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5dbc0d-9f57-46dd-93b0-8b45ff8d545a None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:33.685191 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 201/389] 10.222.0.46 () {58 vars in 1442 bytes} [Thu Jun 23 15:47:33 2022] PUT /identity/v3/projects/27f259ec3bd04186a70598926b09aa56/groups/556c754e31a74b81bc55c99683c91208/roles/e2a5bbf3200a4d4091d72bc7ff78c34d => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 15:47:33.696521 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf9f221-bf11-42a5-a9dd-9bfceb5dbad6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:33.697730 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf9f221-bf11-42a5-a9dd-9bfceb5dbad6 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:33.698157 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf9f221-bf11-42a5-a9dd-9bfceb5dbad6 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:33.698581 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf9f221-bf11-42a5-a9dd-9bfceb5dbad6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:33.711121 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf9f221-bf11-42a5-a9dd-9bfceb5dbad6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 15:47:33.721661 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-9cf9f221-bf11-42a5-a9dd-9bfceb5dbad6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 15:47:33.721661 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:47:33.721661 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:47:33.721661 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:47:33.721661 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:47:33.721661 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:47:33.721661 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:47:33.721661 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:47:33.721661 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:47:33.721661 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:47:33.721661 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:47:33.721661 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:47:33.721661 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 15:47:33.721661 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 15:47:33.721661 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 15:47:33.721661 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 15:47:33.723300 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 15:47:33.723300 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 15:47:33.723300 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 15:47:33.723300 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 15:47:33.723300 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:47:33.723300 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:47:33.723300 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:47:33.723300 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:47:33.723300 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:47:33.723300 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:47:33.723300 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:47:33.723300 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Jun 23 15:47:33.723300 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:47:33.723300 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:47:33.723300 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:47:33.723300 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:47:33.724304 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:47:33.724304 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:47:33.724304 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:47:33.724304 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:47:33.724304 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 15:47:33.724304 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 15:47:33.724304 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 15:47:33.724304 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 15:47:33.724304 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 15:47:33.724304 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 15:47:33.724304 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 15:47:33.724304 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Jun 23 15:47:33.727020 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 189/390] 10.222.0.46 () {56 vars in 1114 bytes} [Thu Jun 23 15:47:33 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 339 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 15:47:33.733403 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5dbc0d-9f57-46dd-93b0-8b45ff8d545a None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:33.881300 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5dbc0d-9f57-46dd-93b0-8b45ff8d545a None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:33.926552 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5c7764-4273-4dde-bb32-0472f34c35c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:33.927302 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5c7764-4273-4dde-bb32-0472f34c35c7 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:33.927588 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5c7764-4273-4dde-bb32-0472f34c35c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:33.927869 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5c7764-4273-4dde-bb32-0472f34c35c7 None admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:33.928657 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5c7764-4273-4dde-bb32-0472f34c35c7 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:47:33.929043 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5c7764-4273-4dde-bb32-0472f34c35c7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:33.931057 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5c7764-4273-4dde-bb32-0472f34c35c7 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:33.941301 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 202/391] 10.222.0.46 () {56 vars in 1111 bytes} [Thu Jun 23 15:47:33 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:47:33.947341 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf9f221-bf11-42a5-a9dd-9bfceb5dbad6 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:34.058378 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf9f221-bf11-42a5-a9dd-9bfceb5dbad6 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:34.059994 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:34.107532 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ab3dc6-d3b1-4c5c-82f2-0a39187c3fbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:34.108398 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ab3dc6-d3b1-4c5c-82f2-0a39187c3fbe None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:34.108685 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ab3dc6-d3b1-4c5c-82f2-0a39187c3fbe None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:34.108959 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ab3dc6-d3b1-4c5c-82f2-0a39187c3fbe None admin] PATH_INFO: `/v3/groups/556c754e31a74b81bc55c99683c91208` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:34.110144 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ab3dc6-d3b1-4c5c-82f2-0a39187c3fbe None admin] RBAC: Authorizing `identity:get_group(group_id=556c754e31a74b81bc55c99683c91208)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:34.111620 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ab3dc6-d3b1-4c5c-82f2-0a39187c3fbe None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:34.113250 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 190/392] 10.222.0.46 () {56 vars in 1180 bytes} [Thu Jun 23 15:47:33 2022] GET /identity/v3/groups/556c754e31a74b81bc55c99683c91208 => generated 226 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:47:34.118868 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5c7764-4273-4dde-bb32-0472f34c35c7 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:34.130640 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:34.227677 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5c7764-4273-4dde-bb32-0472f34c35c7 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:34.261981 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d871aa6-62e4-4c02-bea0-25bb0c86e071 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:34.262881 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d871aa6-62e4-4c02-bea0-25bb0c86e071 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:34.263170 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d871aa6-62e4-4c02-bea0-25bb0c86e071 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:34.263450 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d871aa6-62e4-4c02-bea0-25bb0c86e071 None admin] PATH_INFO: `/v3/projects/27f259ec3bd04186a70598926b09aa56` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:34.264647 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d871aa6-62e4-4c02-bea0-25bb0c86e071 None admin] RBAC: Authorizing `identity:get_project(project_id=27f259ec3bd04186a70598926b09aa56)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:34.266084 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d871aa6-62e4-4c02-bea0-25bb0c86e071 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:34.268144 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 203/393] 10.222.0.46 () {56 vars in 1186 bytes} [Thu Jun 23 15:47:34 2022] GET /identity/v3/projects/27f259ec3bd04186a70598926b09aa56 => generated 298 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:47:34.274332 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ab3dc6-d3b1-4c5c-82f2-0a39187c3fbe None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:34.402622 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ab3dc6-d3b1-4c5c-82f2-0a39187c3fbe None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:34.449805 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63daf88f-9e9d-49f2-915e-d95f9b207382 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:34.450725 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63daf88f-9e9d-49f2-915e-d95f9b207382 None admin] REQUEST_METHOD: `PUT` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:34.451014 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63daf88f-9e9d-49f2-915e-d95f9b207382 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:34.451289 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63daf88f-9e9d-49f2-915e-d95f9b207382 None admin] PATH_INFO: `/v3/projects/27f259ec3bd04186a70598926b09aa56/groups/556c754e31a74b81bc55c99683c91208/roles/d4d557a3ec4e434f8e83499c5fcc7136` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:34.459464 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63daf88f-9e9d-49f2-915e-d95f9b207382 None admin] RBAC: Authorizing `identity:create_grant(project_id=27f259ec3bd04186a70598926b09aa56, group_id=556c754e31a74b81bc55c99683c91208, role_id=d4d557a3ec4e434f8e83499c5fcc7136)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:34.461507 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63daf88f-9e9d-49f2-915e-d95f9b207382 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:34.482974 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 191/394] 10.222.0.46 () {58 vars in 1442 bytes} [Thu Jun 23 15:47:34 2022] PUT /identity/v3/projects/27f259ec3bd04186a70598926b09aa56/groups/556c754e31a74b81bc55c99683c91208/roles/d4d557a3ec4e434f8e83499c5fcc7136 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 15:47:35.013760 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d3c1c5-45a0-40c6-bc43-5cc6a2bb31c2 None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:35.014799 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d3c1c5-45a0-40c6-bc43-5cc6a2bb31c2 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:35.015502 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d3c1c5-45a0-40c6-bc43-5cc6a2bb31c2 None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:35.017510 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 204/395] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jun 23 15:47:35 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 15:47:35.028959 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2622cf-2ca3-4af4-8e8a-e81a312a9667 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:35.029858 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2622cf-2ca3-4af4-8e8a-e81a312a9667 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:35.030356 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2622cf-2ca3-4af4-8e8a-e81a312a9667 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:35.051531 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-4c2622cf-2ca3-4af4-8e8a-e81a312a9667 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:47:35.053060 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-4c2622cf-2ca3-4af4-8e8a-e81a312a9667 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:47:35.084114 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:35.123530 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42b45b3-d466-4828-a9aa-936c2f2341d6 None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:35.123847 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42b45b3-d466-4828-a9aa-936c2f2341d6 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:35.124101 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42b45b3-d466-4828-a9aa-936c2f2341d6 None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:35.126466 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 205/396] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jun 23 15:47:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 15:47:35.130469 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82029a3-629b-4a25-8b24-e11c0c21ab30 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:35.130782 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82029a3-629b-4a25-8b24-e11c0c21ab30 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:35.131066 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82029a3-629b-4a25-8b24-e11c0c21ab30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:35.142526 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 192/397] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jun 23 15:47:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 15:47:35.159998 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-a82029a3-629b-4a25-8b24-e11c0c21ab30 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:47:35.161207 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-a82029a3-629b-4a25-8b24-e11c0c21ab30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:47:35.167760 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2622cf-2ca3-4af4-8e8a-e81a312a9667 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:35.276092 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 206/398] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jun 23 15:47:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 15:47:35.277630 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1976ac-3ee2-4903-b651-21521ce87e03 None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:35.277916 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1976ac-3ee2-4903-b651-21521ce87e03 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:35.278198 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1976ac-3ee2-4903-b651-21521ce87e03 None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:35.280584 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2622cf-2ca3-4af4-8e8a-e81a312a9667 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:35.284670 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 207/399] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jun 23 15:47:35 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 15:47:35.292016 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7d70a0-7ac7-455c-98d2-0bbe56b618eb None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:35.292307 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7d70a0-7ac7-455c-98d2-0bbe56b618eb None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:35.292588 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7d70a0-7ac7-455c-98d2-0bbe56b618eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:35.308259 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:35.316520 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-cd7d70a0-7ac7-455c-98d2-0bbe56b618eb None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:47:35.317992 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-cd7d70a0-7ac7-455c-98d2-0bbe56b618eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:47:35.337140 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef65c42-6559-44ab-9055-45e50f81e51b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:35.337957 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef65c42-6559-44ab-9055-45e50f81e51b None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:35.338244 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef65c42-6559-44ab-9055-45e50f81e51b None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:35.338521 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef65c42-6559-44ab-9055-45e50f81e51b None admin] PATH_INFO: `/v3/roles/member` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:35.350140 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef65c42-6559-44ab-9055-45e50f81e51b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 15:47:35.359293 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-eef65c42-6559-44ab-9055-45e50f81e51b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 23 15:47:35.359293 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:47:35.359293 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:47:35.359293 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:47:35.359293 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:47:35.359293 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:47:35.359293 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:47:35.359293 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:47:35.359293 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:47:35.359293 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:47:35.359293 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:47:35.359293 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:47:35.359293 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 15:47:35.359293 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 15:47:35.359293 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 15:47:35.359293 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 15:47:35.360266 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 15:47:35.360266 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 15:47:35.360266 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 15:47:35.360266 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 15:47:35.360266 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:47:35.360266 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:47:35.360266 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:47:35.360266 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:47:35.360266 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:47:35.360266 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:47:35.360266 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:47:35.360266 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Jun 23 15:47:35.360266 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:47:35.360266 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:47:35.360266 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:47:35.360266 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:47:35.361066 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:47:35.361066 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:47:35.361066 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:47:35.361066 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:47:35.361066 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 15:47:35.361066 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 15:47:35.361066 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 15:47:35.361066 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 15:47:35.361066 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 15:47:35.361066 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 15:47:35.361066 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 23 15:47:35.361066 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Jun 23 15:47:35.361665 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 193/400] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Jun 23 15:47:35 2022] GET /identity/v3/roles/member => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:47:35.362208 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef65c42-6559-44ab-9055-45e50f81e51b None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:35.389204 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:35.406677 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 208/401] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jun 23 15:47:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 15:47:35.407865 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7d70a0-7ac7-455c-98d2-0bbe56b618eb None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:35.495700 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef65c42-6559-44ab-9055-45e50f81e51b None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:35.513681 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7d70a0-7ac7-455c-98d2-0bbe56b618eb None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:35.531057 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36815d04-b71c-4bbd-a51f-8a4c62b8abed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:35.531833 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36815d04-b71c-4bbd-a51f-8a4c62b8abed None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:35.532117 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36815d04-b71c-4bbd-a51f-8a4c62b8abed None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:35.532495 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36815d04-b71c-4bbd-a51f-8a4c62b8abed None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:35.544221 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36815d04-b71c-4bbd-a51f-8a4c62b8abed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 15:47:35.551898 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ad1bd9-704f-4d81-ba1c-60edf43c15e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:35.552820 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ad1bd9-704f-4d81-ba1c-60edf43c15e5 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:35.553101 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-36815d04-b71c-4bbd-a51f-8a4c62b8abed None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 15:47:35.553101 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:47:35.553101 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:47:35.553101 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:47:35.553101 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:47:35.553101 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:47:35.553101 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:47:35.553101 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:47:35.553101 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:47:35.553101 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:47:35.553101 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:47:35.553101 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:47:35.553101 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 15:47:35.553101 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 15:47:35.553101 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 15:47:35.553101 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 15:47:35.554002 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 15:47:35.554002 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 15:47:35.554002 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 15:47:35.554002 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 15:47:35.554002 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:47:35.554002 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:47:35.554002 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:47:35.554002 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:47:35.554002 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:47:35.554002 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:47:35.554002 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:47:35.554002 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Jun 23 15:47:35.554002 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:47:35.554002 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:47:35.554002 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:47:35.554002 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:47:35.554826 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:47:35.554826 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:47:35.554826 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:47:35.554826 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:47:35.554826 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 15:47:35.554826 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 15:47:35.554826 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 15:47:35.554826 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 15:47:35.554826 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 15:47:35.554826 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 15:47:35.554826 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 15:47:35.554826 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Jun 23 15:47:35.556137 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ad1bd9-704f-4d81-ba1c-60edf43c15e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:35.556624 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ad1bd9-704f-4d81-ba1c-60edf43c15e5 None admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:35.557602 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ad1bd9-704f-4d81-ba1c-60edf43c15e5 None admin] RBAC: Adding query filter params (name=member) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:47:35.558113 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ad1bd9-704f-4d81-ba1c-60edf43c15e5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:35.559334 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 194/402] 10.222.0.46 () {56 vars in 1114 bytes} [Thu Jun 23 15:47:35 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:47:35.559999 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ad1bd9-704f-4d81-ba1c-60edf43c15e5 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:35.560257 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36815d04-b71c-4bbd-a51f-8a4c62b8abed None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:35.563167 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:35.570684 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 209/403] 10.222.0.46 () {56 vars in 1101 bytes} [Thu Jun 23 15:47:35 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:47:35.576778 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ad1bd9-704f-4d81-ba1c-60edf43c15e5 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:35.659068 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:35.672543 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36815d04-b71c-4bbd-a51f-8a4c62b8abed None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:35.684072 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ad1bd9-704f-4d81-ba1c-60edf43c15e5 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:35.707806 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0a05ab-d865-4195-9b0f-05c8c3e5d2b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:35.708633 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0a05ab-d865-4195-9b0f-05c8c3e5d2b7 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:35.708897 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0a05ab-d865-4195-9b0f-05c8c3e5d2b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:35.709180 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0a05ab-d865-4195-9b0f-05c8c3e5d2b7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:35.719730 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d696ad14-5d4c-4ae9-8b72-fbd2892caf67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:35.720656 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d696ad14-5d4c-4ae9-8b72-fbd2892caf67 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:35.721152 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d696ad14-5d4c-4ae9-8b72-fbd2892caf67 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:35.721656 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d696ad14-5d4c-4ae9-8b72-fbd2892caf67 None admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:35.721896 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0a05ab-d865-4195-9b0f-05c8c3e5d2b7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 15:47:35.722836 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d696ad14-5d4c-4ae9-8b72-fbd2892caf67 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:47:35.723340 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d696ad14-5d4c-4ae9-8b72-fbd2892caf67 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:35.724916 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d696ad14-5d4c-4ae9-8b72-fbd2892caf67 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:35.730376 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-8f0a05ab-d865-4195-9b0f-05c8c3e5d2b7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 15:47:35.730376 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:47:35.730376 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:47:35.730376 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:47:35.730376 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:47:35.730376 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:47:35.730376 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:47:35.730376 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:47:35.730376 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:47:35.730376 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:47:35.730376 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:47:35.730376 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:47:35.730376 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 15:47:35.730376 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 15:47:35.730376 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 15:47:35.730376 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 15:47:35.731287 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 15:47:35.731287 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 15:47:35.731287 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 15:47:35.731287 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 15:47:35.731287 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:47:35.731287 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:47:35.731287 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:47:35.731287 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:47:35.731287 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:47:35.731287 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:47:35.731287 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:47:35.731287 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Jun 23 15:47:35.731287 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:47:35.731287 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:47:35.731287 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:47:35.731287 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:47:35.732081 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:47:35.732081 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:47:35.732081 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:47:35.732081 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:47:35.732081 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 15:47:35.732081 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 15:47:35.732081 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 15:47:35.732081 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 15:47:35.732081 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 15:47:35.732081 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 15:47:35.732081 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 15:47:35.732081 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Jun 23 15:47:35.736632 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 195/404] 10.222.0.46 () {56 vars in 1096 bytes} [Thu Jun 23 15:47:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:47:35.741292 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0a05ab-d865-4195-9b0f-05c8c3e5d2b7 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:35.741859 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 210/405] 10.222.0.46 () {56 vars in 1111 bytes} [Thu Jun 23 15:47:35 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:47:35.742951 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d696ad14-5d4c-4ae9-8b72-fbd2892caf67 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:35.804088 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:35.846270 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d696ad14-5d4c-4ae9-8b72-fbd2892caf67 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:35.846890 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0a05ab-d865-4195-9b0f-05c8c3e5d2b7 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:35.882992 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9041bf-5d00-46ec-bfd8-d8b8c9c1e45f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:35.884117 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9041bf-5d00-46ec-bfd8-d8b8c9c1e45f None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:35.884249 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06449fba-c8f2-4444-85e7-60f889a64822 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:35.884739 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9041bf-5d00-46ec-bfd8-d8b8c9c1e45f None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:35.885108 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06449fba-c8f2-4444-85e7-60f889a64822 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:35.885395 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06449fba-c8f2-4444-85e7-60f889a64822 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:35.885625 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9041bf-5d00-46ec-bfd8-d8b8c9c1e45f None admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:35.885733 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06449fba-c8f2-4444-85e7-60f889a64822 None admin] PATH_INFO: `/v3/groups/556c754e31a74b81bc55c99683c91208` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:35.886806 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9041bf-5d00-46ec-bfd8-d8b8c9c1e45f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:47:35.886909 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06449fba-c8f2-4444-85e7-60f889a64822 None admin] RBAC: Authorizing `identity:get_group(group_id=556c754e31a74b81bc55c99683c91208)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:35.887413 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9041bf-5d00-46ec-bfd8-d8b8c9c1e45f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:35.888330 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06449fba-c8f2-4444-85e7-60f889a64822 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:35.889203 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9041bf-5d00-46ec-bfd8-d8b8c9c1e45f None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:35.892070 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 196/406] 10.222.0.46 () {56 vars in 1180 bytes} [Thu Jun 23 15:47:35 2022] GET /identity/v3/groups/556c754e31a74b81bc55c99683c91208 => generated 226 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:47:35.892070 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06449fba-c8f2-4444-85e7-60f889a64822 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:35.903910 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:35.905116 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 211/407] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Jun 23 15:47:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:47:35.906158 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9041bf-5d00-46ec-bfd8-d8b8c9c1e45f None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:36.010823 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06449fba-c8f2-4444-85e7-60f889a64822 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:36.012759 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9041bf-5d00-46ec-bfd8-d8b8c9c1e45f None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:36.036637 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:36.048736 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9a5d02-b0fb-4101-a99b-8e048cc29c8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:36.049702 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9a5d02-b0fb-4101-a99b-8e048cc29c8c None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:36.049991 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9a5d02-b0fb-4101-a99b-8e048cc29c8c None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:36.050268 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9a5d02-b0fb-4101-a99b-8e048cc29c8c None admin] PATH_INFO: `/v3/projects/2240bbdc36ed4e59a7454b287b5891ab` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:36.051485 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9a5d02-b0fb-4101-a99b-8e048cc29c8c None admin] RBAC: Authorizing `identity:get_project(project_id=2240bbdc36ed4e59a7454b287b5891ab)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:36.052831 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343da23f-e8e2-4897-be8b-038383a639a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:36.053291 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9a5d02-b0fb-4101-a99b-8e048cc29c8c None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:36.054378 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343da23f-e8e2-4897-be8b-038383a639a5 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:36.054848 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343da23f-e8e2-4897-be8b-038383a639a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:36.055028 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 212/408] 10.222.0.46 () {56 vars in 1186 bytes} [Thu Jun 23 15:47:35 2022] GET /identity/v3/projects/2240bbdc36ed4e59a7454b287b5891ab => generated 302 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:47:36.055471 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343da23f-e8e2-4897-be8b-038383a639a5 None admin] PATH_INFO: `/v3/groups/556c754e31a74b81bc55c99683c91208` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:36.055770 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9a5d02-b0fb-4101-a99b-8e048cc29c8c None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:36.057116 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343da23f-e8e2-4897-be8b-038383a639a5 None admin] RBAC: Authorizing `identity:get_group(group_id=556c754e31a74b81bc55c99683c91208)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:36.058887 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343da23f-e8e2-4897-be8b-038383a639a5 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:36.061150 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 197/409] 10.222.0.46 () {56 vars in 1180 bytes} [Thu Jun 23 15:47:35 2022] GET /identity/v3/groups/556c754e31a74b81bc55c99683c91208 => generated 226 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:47:36.062227 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343da23f-e8e2-4897-be8b-038383a639a5 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:36.148501 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:36.159436 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9a5d02-b0fb-4101-a99b-8e048cc29c8c None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:36.175987 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343da23f-e8e2-4897-be8b-038383a639a5 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:36.193366 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998bad8d-328a-4b26-b6d8-78b83216e35e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:36.194231 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998bad8d-328a-4b26-b6d8-78b83216e35e None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:36.195072 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998bad8d-328a-4b26-b6d8-78b83216e35e None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:36.195350 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998bad8d-328a-4b26-b6d8-78b83216e35e None admin] PATH_INFO: `/v3/groups/47834ccd8b0046b2920f4352334f4694` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:36.196531 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998bad8d-328a-4b26-b6d8-78b83216e35e None admin] RBAC: Authorizing `identity:get_group(group_id=47834ccd8b0046b2920f4352334f4694)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:36.197981 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998bad8d-328a-4b26-b6d8-78b83216e35e None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:36.204626 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 213/410] 10.222.0.46 () {56 vars in 1180 bytes} [Thu Jun 23 15:47:36 2022] GET /identity/v3/groups/47834ccd8b0046b2920f4352334f4694 => generated 229 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:47:36.204626 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998bad8d-328a-4b26-b6d8-78b83216e35e None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:36.220888 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627302fa-0829-4e5e-8873-4ad000d5a2b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:36.222297 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627302fa-0829-4e5e-8873-4ad000d5a2b3 None admin] REQUEST_METHOD: `PUT` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:36.222829 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627302fa-0829-4e5e-8873-4ad000d5a2b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:36.223302 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627302fa-0829-4e5e-8873-4ad000d5a2b3 None admin] PATH_INFO: `/v3/projects/2240bbdc36ed4e59a7454b287b5891ab/groups/556c754e31a74b81bc55c99683c91208/roles/e2a5bbf3200a4d4091d72bc7ff78c34d` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:36.225579 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627302fa-0829-4e5e-8873-4ad000d5a2b3 None admin] RBAC: Authorizing `identity:create_grant(project_id=2240bbdc36ed4e59a7454b287b5891ab, group_id=556c754e31a74b81bc55c99683c91208, role_id=e2a5bbf3200a4d4091d72bc7ff78c34d)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:36.227339 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627302fa-0829-4e5e-8873-4ad000d5a2b3 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:36.245259 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 198/411] 10.222.0.46 () {58 vars in 1442 bytes} [Thu Jun 23 15:47:36 2022] PUT /identity/v3/projects/2240bbdc36ed4e59a7454b287b5891ab/groups/556c754e31a74b81bc55c99683c91208/roles/e2a5bbf3200a4d4091d72bc7ff78c34d => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 15:47:36.247400 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627302fa-0829-4e5e-8873-4ad000d5a2b3 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:36.270816 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:36.368879 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998bad8d-328a-4b26-b6d8-78b83216e35e None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:36.384683 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627302fa-0829-4e5e-8873-4ad000d5a2b3 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:36.428124 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2036d92-bd61-41c4-8cd3-4e881358c95b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:36.429564 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2036d92-bd61-41c4-8cd3-4e881358c95b None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:36.429852 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2036d92-bd61-41c4-8cd3-4e881358c95b None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:36.430131 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2036d92-bd61-41c4-8cd3-4e881358c95b None admin] PATH_INFO: `/v3/projects/2240bbdc36ed4e59a7454b287b5891ab` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:36.431400 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2036d92-bd61-41c4-8cd3-4e881358c95b None admin] RBAC: Authorizing `identity:get_project(project_id=2240bbdc36ed4e59a7454b287b5891ab)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:36.432919 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2036d92-bd61-41c4-8cd3-4e881358c95b None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:36.433566 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b7624c-195f-4ebd-8303-8d473d997ee3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:36.434941 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 214/412] 10.222.0.46 () {56 vars in 1186 bytes} [Thu Jun 23 15:47:36 2022] GET /identity/v3/projects/2240bbdc36ed4e59a7454b287b5891ab => generated 302 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:47:36.435312 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b7624c-195f-4ebd-8303-8d473d997ee3 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:36.435858 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b7624c-195f-4ebd-8303-8d473d997ee3 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:36.436151 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7f32ec-c872-4286-b6dd-8931d0bd5eb3 None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:36.436443 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7f32ec-c872-4286-b6dd-8931d0bd5eb3 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:36.436725 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7f32ec-c872-4286-b6dd-8931d0bd5eb3 None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:36.436910 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b7624c-195f-4ebd-8303-8d473d997ee3 None admin] PATH_INFO: `/v3/projects/110dee34db4740f989f715e757ee809c` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:36.438482 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b7624c-195f-4ebd-8303-8d473d997ee3 None admin] RBAC: Authorizing `identity:get_project(project_id=110dee34db4740f989f715e757ee809c)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:36.440394 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b7624c-195f-4ebd-8303-8d473d997ee3 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:36.442276 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 215/413] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jun 23 15:47:36 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 15:47:36.446834 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7f32ec-c872-4286-b6dd-8931d0bd5eb3 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:36.447333 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 199/414] 10.222.0.46 () {56 vars in 1186 bytes} [Thu Jun 23 15:47:36 2022] GET /identity/v3/projects/110dee34db4740f989f715e757ee809c => generated 344 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:47:36.450123 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6096b23-5709-4b42-9c81-35d60dcc05c4 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:36.450588 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6096b23-5709-4b42-9c81-35d60dcc05c4 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:36.451034 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6096b23-5709-4b42-9c81-35d60dcc05c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:36.462461 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:36.480724 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-d6096b23-5709-4b42-9c81-35d60dcc05c4 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:47:36.482510 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-d6096b23-5709-4b42-9c81-35d60dcc05c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:47:36.567260 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 200/415] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jun 23 15:47:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 15:47:36.574714 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6096b23-5709-4b42-9c81-35d60dcc05c4 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:36.621304 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7f32ec-c872-4286-b6dd-8931d0bd5eb3 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:36.650259 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:36.662305 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0ef903-42a0-47f7-ad29-dfa9ee24cc29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:36.663564 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0ef903-42a0-47f7-ad29-dfa9ee24cc29 None admin] REQUEST_METHOD: `PUT` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:36.664031 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0ef903-42a0-47f7-ad29-dfa9ee24cc29 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:36.664485 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0ef903-42a0-47f7-ad29-dfa9ee24cc29 None admin] PATH_INFO: `/v3/projects/2240bbdc36ed4e59a7454b287b5891ab/groups/556c754e31a74b81bc55c99683c91208/roles/d4d557a3ec4e434f8e83499c5fcc7136` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:36.667495 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0ef903-42a0-47f7-ad29-dfa9ee24cc29 None admin] RBAC: Authorizing `identity:create_grant(project_id=2240bbdc36ed4e59a7454b287b5891ab, group_id=556c754e31a74b81bc55c99683c91208, role_id=d4d557a3ec4e434f8e83499c5fcc7136)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:36.669549 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0ef903-42a0-47f7-ad29-dfa9ee24cc29 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:36.687651 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 216/416] 10.222.0.46 () {58 vars in 1442 bytes} [Thu Jun 23 15:47:36 2022] PUT /identity/v3/projects/2240bbdc36ed4e59a7454b287b5891ab/groups/556c754e31a74b81bc55c99683c91208/roles/d4d557a3ec4e434f8e83499c5fcc7136 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 15:47:36.688957 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0ef903-42a0-47f7-ad29-dfa9ee24cc29 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:36.698544 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6096b23-5709-4b42-9c81-35d60dcc05c4 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:36.757618 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff7fa7b-a334-455c-aa66-e8b06bd80ca8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:36.758638 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff7fa7b-a334-455c-aa66-e8b06bd80ca8 None admin] REQUEST_METHOD: `PUT` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:36.759057 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff7fa7b-a334-455c-aa66-e8b06bd80ca8 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:36.759663 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff7fa7b-a334-455c-aa66-e8b06bd80ca8 None admin] PATH_INFO: `/v3/projects/110dee34db4740f989f715e757ee809c/groups/47834ccd8b0046b2920f4352334f4694/roles/812d3c567c52469cb2b35709fbdc857c` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:36.762286 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff7fa7b-a334-455c-aa66-e8b06bd80ca8 None admin] RBAC: Authorizing `identity:create_grant(project_id=110dee34db4740f989f715e757ee809c, group_id=47834ccd8b0046b2920f4352334f4694, role_id=812d3c567c52469cb2b35709fbdc857c)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:36.764659 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff7fa7b-a334-455c-aa66-e8b06bd80ca8 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:36.767810 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:36.786781 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 201/417] 10.222.0.46 () {58 vars in 1442 bytes} [Thu Jun 23 15:47:36 2022] PUT /identity/v3/projects/110dee34db4740f989f715e757ee809c/groups/47834ccd8b0046b2920f4352334f4694/roles/812d3c567c52469cb2b35709fbdc857c => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 15:47:36.842927 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0ef903-42a0-47f7-ad29-dfa9ee24cc29 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:36.896104 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc922d5-02ba-49e3-806d-f3b8e00d53f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:36.897009 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc922d5-02ba-49e3-806d-f3b8e00d53f6 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:36.897293 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc922d5-02ba-49e3-806d-f3b8e00d53f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:36.897619 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc922d5-02ba-49e3-806d-f3b8e00d53f6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:36.916324 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc922d5-02ba-49e3-806d-f3b8e00d53f6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 15:47:36.924459 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-fbc922d5-02ba-49e3-806d-f3b8e00d53f6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 23 15:47:36.924459 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:47:36.924459 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:47:36.924459 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:47:36.924459 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:47:36.924459 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:47:36.924459 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:47:36.924459 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:47:36.924459 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:47:36.924459 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:47:36.924459 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:47:36.924459 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:47:36.924459 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 15:47:36.924459 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 15:47:36.924459 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 15:47:36.924459 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 15:47:36.925614 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 15:47:36.925614 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 15:47:36.925614 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 15:47:36.925614 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 15:47:36.925614 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:47:36.925614 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:47:36.925614 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:47:36.925614 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:47:36.925614 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:47:36.925614 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:47:36.925614 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:47:36.925614 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Jun 23 15:47:36.925614 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:47:36.925614 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:47:36.925614 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:47:36.925614 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:47:36.926708 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:47:36.926708 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:47:36.926708 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:47:36.926708 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:47:36.926708 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 15:47:36.926708 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 15:47:36.926708 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 15:47:36.926708 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 15:47:36.926708 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 15:47:36.926708 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 15:47:36.926708 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 23 15:47:36.926708 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Jun 23 15:47:36.926708 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 217/418] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Jun 23 15:47:36 2022] GET /identity/v3/roles/member => generated 84 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:47:36.932060 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff7fa7b-a334-455c-aa66-e8b06bd80ca8 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:37.066851 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3c7408-4417-4738-a307-8ec2be8783d5 None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:37.067362 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff7fa7b-a334-455c-aa66-e8b06bd80ca8 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:37.068190 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3c7408-4417-4738-a307-8ec2be8783d5 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:37.068691 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3c7408-4417-4738-a307-8ec2be8783d5 None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:37.070895 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 218/419] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jun 23 15:47:37 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 15:47:37.086334 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ca2510-67a9-4852-b227-dc8333c36367 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:37.086627 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ca2510-67a9-4852-b227-dc8333c36367 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:37.086928 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ca2510-67a9-4852-b227-dc8333c36367 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:37.109344 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad281dff-68e9-4a86-835d-623591379dee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:37.110038 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-f3ca2510-67a9-4852-b227-dc8333c36367 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:47:37.110642 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad281dff-68e9-4a86-835d-623591379dee None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:37.111183 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad281dff-68e9-4a86-835d-623591379dee None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:37.111285 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-f3ca2510-67a9-4852-b227-dc8333c36367 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:47:37.111761 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad281dff-68e9-4a86-835d-623591379dee None admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:37.112723 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad281dff-68e9-4a86-835d-623591379dee None admin] RBAC: Adding query filter params (name=member) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:47:37.113240 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad281dff-68e9-4a86-835d-623591379dee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:37.114876 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad281dff-68e9-4a86-835d-623591379dee None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:37.134853 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 202/420] 10.222.0.46 () {56 vars in 1101 bytes} [Thu Jun 23 15:47:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:47:37.140019 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad281dff-68e9-4a86-835d-623591379dee None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:37.159097 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:37.187437 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 219/421] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jun 23 15:47:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 15:47:37.234793 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ca2510-67a9-4852-b227-dc8333c36367 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:37.247980 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:37.325107 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad281dff-68e9-4a86-835d-623591379dee None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:37.365519 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e369677c-07b2-4238-8a8e-8b1cc3373229 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:37.366951 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e369677c-07b2-4238-8a8e-8b1cc3373229 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:37.367247 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e369677c-07b2-4238-8a8e-8b1cc3373229 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:37.367530 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e369677c-07b2-4238-8a8e-8b1cc3373229 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:37.425414 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ca2510-67a9-4852-b227-dc8333c36367 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:37.442716 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e369677c-07b2-4238-8a8e-8b1cc3373229 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:37.444215 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e369677c-07b2-4238-8a8e-8b1cc3373229 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:37.495441 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b07b90-035e-4259-b828-549fa006c5f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:37.496270 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b07b90-035e-4259-b828-549fa006c5f6 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:37.496547 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b07b90-035e-4259-b828-549fa006c5f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:37.496820 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b07b90-035e-4259-b828-549fa006c5f6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:37.508987 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:37.520049 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b07b90-035e-4259-b828-549fa006c5f6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 15:47:37.536640 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-d0b07b90-035e-4259-b828-549fa006c5f6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 15:47:37.536640 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:47:37.536640 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:47:37.536640 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:47:37.536640 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:47:37.536640 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:47:37.536640 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:47:37.536640 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:47:37.536640 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:47:37.536640 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:47:37.536640 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:47:37.536640 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:47:37.536640 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 15:47:37.536640 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 15:47:37.536640 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 15:47:37.536640 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 15:47:37.537578 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 15:47:37.537578 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 15:47:37.537578 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 15:47:37.537578 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 15:47:37.537578 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:47:37.537578 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:47:37.537578 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:47:37.537578 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:47:37.537578 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:47:37.537578 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:47:37.537578 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:47:37.537578 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Jun 23 15:47:37.537578 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:47:37.537578 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:47:37.537578 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:47:37.537578 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:47:37.538978 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:47:37.538978 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:47:37.538978 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:47:37.538978 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:47:37.538978 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 15:47:37.538978 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 15:47:37.538978 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 15:47:37.538978 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 15:47:37.538978 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 15:47:37.538978 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 15:47:37.538978 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 15:47:37.538978 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Jun 23 15:47:37.547043 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 220/422] 10.222.0.46 () {56 vars in 1114 bytes} [Thu Jun 23 15:47:37 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 316 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:47:37.547958 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b07b90-035e-4259-b828-549fa006c5f6 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:37.549850 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 203/423] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Jun 23 15:47:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 23 15:47:37.563324 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e369677c-07b2-4238-8a8e-8b1cc3373229 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:37.597130 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:37.666321 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b07b90-035e-4259-b828-549fa006c5f6 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:37.701953 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b56c86-96cd-4241-b9f4-b3b6e1f8bdd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:37.702542 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b56c86-96cd-4241-b9f4-b3b6e1f8bdd7 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:37.702854 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b56c86-96cd-4241-b9f4-b3b6e1f8bdd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:37.703129 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b56c86-96cd-4241-b9f4-b3b6e1f8bdd7 None admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:37.703887 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b56c86-96cd-4241-b9f4-b3b6e1f8bdd7 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:47:37.704191 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b56c86-96cd-4241-b9f4-b3b6e1f8bdd7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:37.705578 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b56c86-96cd-4241-b9f4-b3b6e1f8bdd7 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:37.714914 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 221/424] 10.222.0.46 () {56 vars in 1111 bytes} [Thu Jun 23 15:47:37 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:47:37.720286 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b56c86-96cd-4241-b9f4-b3b6e1f8bdd7 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:37.779730 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e369677c-07b2-4238-8a8e-8b1cc3373229 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:37.823556 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b56c86-96cd-4241-b9f4-b3b6e1f8bdd7 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:37.826667 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:37.865139 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4add965a-1a14-4e5c-bf73-4122e516da09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:37.866008 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4add965a-1a14-4e5c-bf73-4122e516da09 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:37.866853 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4add965a-1a14-4e5c-bf73-4122e516da09 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:37.867129 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4add965a-1a14-4e5c-bf73-4122e516da09 None admin] PATH_INFO: `/v3/projects/27f259ec3bd04186a70598926b09aa56` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:37.869170 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4add965a-1a14-4e5c-bf73-4122e516da09 None admin] RBAC: Authorizing `identity:get_project(project_id=27f259ec3bd04186a70598926b09aa56)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:37.870607 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4add965a-1a14-4e5c-bf73-4122e516da09 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:37.872644 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 204/425] 10.222.0.46 () {56 vars in 1186 bytes} [Thu Jun 23 15:47:37 2022] GET /identity/v3/projects/27f259ec3bd04186a70598926b09aa56 => generated 298 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:47:37.877923 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4add965a-1a14-4e5c-bf73-4122e516da09 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:37.888303 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923ba48c-2968-4eda-941c-19357d6f5b17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:37.888976 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923ba48c-2968-4eda-941c-19357d6f5b17 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:37.889262 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923ba48c-2968-4eda-941c-19357d6f5b17 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:37.889543 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923ba48c-2968-4eda-941c-19357d6f5b17 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:37.904295 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:38.000955 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4add965a-1a14-4e5c-bf73-4122e516da09 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:38.046265 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923ba48c-2968-4eda-941c-19357d6f5b17 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:38.047776 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923ba48c-2968-4eda-941c-19357d6f5b17 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:38.067460 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ada114-91eb-47fa-960d-13f8f78979d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:38.068349 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ada114-91eb-47fa-960d-13f8f78979d9 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:38.068641 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ada114-91eb-47fa-960d-13f8f78979d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:38.068920 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ada114-91eb-47fa-960d-13f8f78979d9 None admin] PATH_INFO: `/v3/groups/556c754e31a74b81bc55c99683c91208` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:38.070829 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ada114-91eb-47fa-960d-13f8f78979d9 None admin] RBAC: Authorizing `identity:get_group(group_id=556c754e31a74b81bc55c99683c91208)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:38.073128 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ada114-91eb-47fa-960d-13f8f78979d9 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:38.074821 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 205/426] 10.222.0.46 () {56 vars in 1180 bytes} [Thu Jun 23 15:47:37 2022] GET /identity/v3/groups/556c754e31a74b81bc55c99683c91208 => generated 226 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 15:47:38.083668 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ada114-91eb-47fa-960d-13f8f78979d9 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:38.163649 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:38.168484 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 222/427] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Jun 23 15:47:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 448 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 15:47:38.173706 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923ba48c-2968-4eda-941c-19357d6f5b17 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:38.297389 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:38.299582 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923ba48c-2968-4eda-941c-19357d6f5b17 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:38.326245 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ada114-91eb-47fa-960d-13f8f78979d9 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:38.341892 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c405c6-9d25-4338-b8ff-0c4522ee1389 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:38.342801 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c405c6-9d25-4338-b8ff-0c4522ee1389 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:38.343134 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c405c6-9d25-4338-b8ff-0c4522ee1389 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:38.343419 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c405c6-9d25-4338-b8ff-0c4522ee1389 None admin] PATH_INFO: `/v3/projects/27f259ec3bd04186a70598926b09aa56` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:38.344657 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c405c6-9d25-4338-b8ff-0c4522ee1389 None admin] RBAC: Authorizing `identity:get_project(project_id=27f259ec3bd04186a70598926b09aa56)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:38.346133 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c405c6-9d25-4338-b8ff-0c4522ee1389 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:38.348251 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 223/428] 10.222.0.46 () {56 vars in 1186 bytes} [Thu Jun 23 15:47:38 2022] GET /identity/v3/projects/27f259ec3bd04186a70598926b09aa56 => generated 298 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 15:47:38.354451 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c405c6-9d25-4338-b8ff-0c4522ee1389 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:38.398242 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006d86a0-f8fe-4187-a823-6e7e8105535e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:38.399306 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006d86a0-f8fe-4187-a823-6e7e8105535e None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:38.399782 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006d86a0-f8fe-4187-a823-6e7e8105535e None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:38.400736 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006d86a0-f8fe-4187-a823-6e7e8105535e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:38.402730 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006d86a0-f8fe-4187-a823-6e7e8105535e None admin] RBAC: Adding query filter params (group.id=556c754e31a74b81bc55c99683c91208, role.id=e2a5bbf3200a4d4091d72bc7ff78c34d, scope.project.id=27f259ec3bd04186a70598926b09aa56) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:47:38.403421 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006d86a0-f8fe-4187-a823-6e7e8105535e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:38.406259 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006d86a0-f8fe-4187-a823-6e7e8105535e None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:38.423065 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 206/429] 10.222.0.46 () {56 vars in 1376 bytes} [Thu Jun 23 15:47:38 2022] GET /identity/v3/role_assignments?group.id=556c754e31a74b81bc55c99683c91208&scope.project.id=27f259ec3bd04186a70598926b09aa56&role.id=e2a5bbf3200a4d4091d72bc7ff78c34d => generated 611 bytes in 335 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:47:38.473808 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:38.530202 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c405c6-9d25-4338-b8ff-0c4522ee1389 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:38.583587 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3026118-fdfb-41ef-8402-6046fa4d2141 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:38.584485 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3026118-fdfb-41ef-8402-6046fa4d2141 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:38.584764 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3026118-fdfb-41ef-8402-6046fa4d2141 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:38.585595 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3026118-fdfb-41ef-8402-6046fa4d2141 None admin] PATH_INFO: `/v3/groups/556c754e31a74b81bc55c99683c91208` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:38.586911 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3026118-fdfb-41ef-8402-6046fa4d2141 None admin] RBAC: Authorizing `identity:get_group(group_id=556c754e31a74b81bc55c99683c91208)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:38.588502 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3026118-fdfb-41ef-8402-6046fa4d2141 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:38.590353 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 224/430] 10.222.0.46 () {56 vars in 1180 bytes} [Thu Jun 23 15:47:38 2022] GET /identity/v3/groups/556c754e31a74b81bc55c99683c91208 => generated 226 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:47:38.596038 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006d86a0-f8fe-4187-a823-6e7e8105535e None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:38.729710 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db0746e-61f1-435a-b988-ebf8c1ae8299 None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:38.730093 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db0746e-61f1-435a-b988-ebf8c1ae8299 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:38.730272 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db0746e-61f1-435a-b988-ebf8c1ae8299 None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:38.732094 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 225/431] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jun 23 15:47:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 15:47:38.733169 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006d86a0-f8fe-4187-a823-6e7e8105535e None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:38.737812 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e01ecf-d5b2-42ea-9d5a-12651b613762 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:38.738108 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e01ecf-d5b2-42ea-9d5a-12651b613762 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:38.738379 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e01ecf-d5b2-42ea-9d5a-12651b613762 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:38.759392 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-d5e01ecf-d5b2-42ea-9d5a-12651b613762 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:47:38.760831 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-d5e01ecf-d5b2-42ea-9d5a-12651b613762 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:47:38.802812 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b9c255-de65-4e77-9d30-4461c98044d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:38.803498 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b9c255-de65-4e77-9d30-4461c98044d9 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:38.804317 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b9c255-de65-4e77-9d30-4461c98044d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:38.804595 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b9c255-de65-4e77-9d30-4461c98044d9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:38.805443 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b9c255-de65-4e77-9d30-4461c98044d9 None admin] RBAC: Adding query filter params (group.id=556c754e31a74b81bc55c99683c91208, role.id=d4d557a3ec4e434f8e83499c5fcc7136, scope.project.id=27f259ec3bd04186a70598926b09aa56) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:47:38.805752 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b9c255-de65-4e77-9d30-4461c98044d9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:38.807234 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b9c255-de65-4e77-9d30-4461c98044d9 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:38.820552 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 207/432] 10.222.0.46 () {56 vars in 1376 bytes} [Thu Jun 23 15:47:38 2022] GET /identity/v3/role_assignments?group.id=556c754e31a74b81bc55c99683c91208&scope.project.id=27f259ec3bd04186a70598926b09aa56&role.id=d4d557a3ec4e434f8e83499c5fcc7136 => generated 611 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:47:38.849340 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 226/433] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jun 23 15:47:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 15:47:38.881127 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b9c255-de65-4e77-9d30-4461c98044d9 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:38.916919 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:39.039477 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b9c255-de65-4e77-9d30-4461c98044d9 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:39.078773 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ed921a-0b60-4c21-acfe-c4838660d89b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:39.079698 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ed921a-0b60-4c21-acfe-c4838660d89b None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:39.079987 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ed921a-0b60-4c21-acfe-c4838660d89b None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:39.080263 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ed921a-0b60-4c21-acfe-c4838660d89b None admin] PATH_INFO: `/v3/roles/member` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:39.095850 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ed921a-0b60-4c21-acfe-c4838660d89b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 15:47:39.104478 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-83ed921a-0b60-4c21-acfe-c4838660d89b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 23 15:47:39.104478 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:47:39.104478 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:47:39.104478 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:47:39.104478 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:47:39.104478 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:47:39.104478 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:47:39.104478 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:47:39.104478 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:47:39.104478 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:47:39.104478 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:47:39.104478 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:47:39.104478 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 15:47:39.104478 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 15:47:39.104478 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 15:47:39.104478 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 15:47:39.105399 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 15:47:39.105399 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 15:47:39.105399 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 15:47:39.105399 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 15:47:39.105399 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:47:39.105399 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:47:39.105399 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:47:39.105399 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:47:39.105399 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:47:39.105399 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:47:39.105399 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:47:39.105399 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Jun 23 15:47:39.105399 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:47:39.105399 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:47:39.105399 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:47:39.105399 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:47:39.106459 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:47:39.106459 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:47:39.106459 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:47:39.106459 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:47:39.106459 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 15:47:39.106459 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 15:47:39.106459 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 15:47:39.106459 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 15:47:39.106459 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 15:47:39.106459 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 15:47:39.106459 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 23 15:47:39.106459 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Jun 23 15:47:39.106459 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 208/434] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Jun 23 15:47:38 2022] GET /identity/v3/roles/member => generated 84 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:47:39.111475 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e01ecf-d5b2-42ea-9d5a-12651b613762 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:39.193850 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:39.219779 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def0099a-3cf3-488f-a130-60f7c55249f0 None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:39.220067 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def0099a-3cf3-488f-a130-60f7c55249f0 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:39.220513 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def0099a-3cf3-488f-a130-60f7c55249f0 None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:39.222384 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 209/435] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jun 23 15:47:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 15:47:39.228738 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40d2b20-7d51-4de4-8bd9-6c0e6229fee5 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:39.229741 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40d2b20-7d51-4de4-8bd9-6c0e6229fee5 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:39.230031 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40d2b20-7d51-4de4-8bd9-6c0e6229fee5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:39.246970 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e01ecf-d5b2-42ea-9d5a-12651b613762 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:39.257558 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-f40d2b20-7d51-4de4-8bd9-6c0e6229fee5 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:47:39.258863 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-f40d2b20-7d51-4de4-8bd9-6c0e6229fee5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:47:39.297315 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f642ab9-3246-4e64-b5bc-0b9e04215d7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:39.298005 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f642ab9-3246-4e64-b5bc-0b9e04215d7d None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:39.298289 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f642ab9-3246-4e64-b5bc-0b9e04215d7d None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:39.298563 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f642ab9-3246-4e64-b5bc-0b9e04215d7d None admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:39.299387 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f642ab9-3246-4e64-b5bc-0b9e04215d7d None admin] RBAC: Adding query filter params (name=member) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:47:39.299695 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f642ab9-3246-4e64-b5bc-0b9e04215d7d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:39.301091 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f642ab9-3246-4e64-b5bc-0b9e04215d7d None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:39.311477 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 227/436] 10.222.0.46 () {56 vars in 1101 bytes} [Thu Jun 23 15:47:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:47:39.318993 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f642ab9-3246-4e64-b5bc-0b9e04215d7d None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:39.358677 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 210/437] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jun 23 15:47:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 15:47:39.361684 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3ae43b-a669-4d69-a9cc-2c76c95898b8 None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:39.361985 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3ae43b-a669-4d69-a9cc-2c76c95898b8 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:39.362265 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3ae43b-a669-4d69-a9cc-2c76c95898b8 None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:39.363622 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 211/438] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jun 23 15:47:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 15:47:39.369318 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0901451-59bb-463f-a2c9-e91298edad34 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:39.369779 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0901451-59bb-463f-a2c9-e91298edad34 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:39.370237 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0901451-59bb-463f-a2c9-e91298edad34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:39.388238 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-f0901451-59bb-463f-a2c9-e91298edad34 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:47:39.389432 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-f0901451-59bb-463f-a2c9-e91298edad34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:47:39.392589 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:39.459922 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f642ab9-3246-4e64-b5bc-0b9e04215d7d None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:39.471432 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 212/439] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jun 23 15:47:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 15:47:39.472290 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0901451-59bb-463f-a2c9-e91298edad34 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:39.486272 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:39.495848 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17dd3099-c7fe-474f-aca3-599612fa4e84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:39.496505 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dd3099-c7fe-474f-aca3-599612fa4e84 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:39.496793 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dd3099-c7fe-474f-aca3-599612fa4e84 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:39.497074 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dd3099-c7fe-474f-aca3-599612fa4e84 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:39.568533 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17dd3099-c7fe-474f-aca3-599612fa4e84 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:39.570591 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17dd3099-c7fe-474f-aca3-599612fa4e84 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:39.583836 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0901451-59bb-463f-a2c9-e91298edad34 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:39.623929 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f53403-1b4b-40a8-a236-fd436f46eb5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:39.624663 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f53403-1b4b-40a8-a236-fd436f46eb5a None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:39.624950 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f53403-1b4b-40a8-a236-fd436f46eb5a None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:39.625228 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f53403-1b4b-40a8-a236-fd436f46eb5a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:39.633515 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:39.636706 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f53403-1b4b-40a8-a236-fd436f46eb5a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 15:47:39.645509 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-51f53403-1b4b-40a8-a236-fd436f46eb5a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 15:47:39.645509 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:47:39.645509 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:47:39.645509 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:47:39.645509 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:47:39.645509 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:47:39.645509 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:47:39.645509 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:47:39.645509 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:47:39.645509 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:47:39.645509 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:47:39.645509 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:47:39.645509 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 15:47:39.645509 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 15:47:39.645509 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 15:47:39.645509 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 15:47:39.646576 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 15:47:39.646576 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 15:47:39.646576 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 15:47:39.646576 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 15:47:39.646576 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:47:39.646576 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:47:39.646576 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:47:39.646576 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:47:39.646576 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:47:39.646576 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:47:39.646576 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:47:39.646576 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Jun 23 15:47:39.646576 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:47:39.646576 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:47:39.646576 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:47:39.646576 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:47:39.647478 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:47:39.647478 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:47:39.647478 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:47:39.647478 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:47:39.647478 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 15:47:39.647478 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 15:47:39.647478 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 15:47:39.647478 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 15:47:39.647478 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 15:47:39.647478 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 15:47:39.647478 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 15:47:39.647478 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Jun 23 15:47:39.647478 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 213/440] 10.222.0.46 () {56 vars in 1114 bytes} [Thu Jun 23 15:47:39 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:47:39.648130 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f53403-1b4b-40a8-a236-fd436f46eb5a None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:39.680057 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 228/441] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Jun 23 15:47:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 15:47:39.680912 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17dd3099-c7fe-474f-aca3-599612fa4e84 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:39.708519 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:39.752410 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f53403-1b4b-40a8-a236-fd436f46eb5a None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:39.783833 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17dd3099-c7fe-474f-aca3-599612fa4e84 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:39.785784 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588a7111-453b-4325-b094-c35009987bea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:39.786588 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588a7111-453b-4325-b094-c35009987bea None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:39.786903 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588a7111-453b-4325-b094-c35009987bea None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:39.787183 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588a7111-453b-4325-b094-c35009987bea None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:39.798150 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588a7111-453b-4325-b094-c35009987bea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 15:47:39.806627 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-588a7111-453b-4325-b094-c35009987bea None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 15:47:39.806627 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:47:39.806627 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:47:39.806627 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:47:39.806627 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:47:39.806627 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:47:39.806627 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:47:39.806627 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:47:39.806627 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:47:39.806627 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:47:39.806627 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:47:39.806627 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:47:39.806627 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 15:47:39.806627 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 15:47:39.806627 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 15:47:39.806627 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 15:47:39.807647 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 15:47:39.807647 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 15:47:39.807647 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 15:47:39.807647 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 15:47:39.807647 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:47:39.807647 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:47:39.807647 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:47:39.807647 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:47:39.807647 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:47:39.807647 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:47:39.807647 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:47:39.807647 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Jun 23 15:47:39.807647 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:47:39.807647 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:47:39.807647 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:47:39.807647 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:47:39.808459 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:47:39.808459 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:47:39.808459 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:47:39.808459 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:47:39.808459 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 15:47:39.808459 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 15:47:39.808459 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 15:47:39.808459 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 15:47:39.808459 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 15:47:39.808459 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 15:47:39.808459 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 15:47:39.808459 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Jun 23 15:47:39.808459 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 214/442] 10.222.0.46 () {56 vars in 1096 bytes} [Thu Jun 23 15:47:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:47:39.809121 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588a7111-453b-4325-b094-c35009987bea None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:39.820484 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d262c39d-6a2e-46fc-8f10-37719c0d46ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:39.821157 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d262c39d-6a2e-46fc-8f10-37719c0d46ba None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:39.821439 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d262c39d-6a2e-46fc-8f10-37719c0d46ba None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:39.821727 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d262c39d-6a2e-46fc-8f10-37719c0d46ba None admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:39.822479 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d262c39d-6a2e-46fc-8f10-37719c0d46ba None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:47:39.822815 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d262c39d-6a2e-46fc-8f10-37719c0d46ba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:39.824191 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d262c39d-6a2e-46fc-8f10-37719c0d46ba None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:39.832385 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 229/443] 10.222.0.46 () {56 vars in 1111 bytes} [Thu Jun 23 15:47:39 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:47:39.833180 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d262c39d-6a2e-46fc-8f10-37719c0d46ba None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:39.868621 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:39.910090 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588a7111-453b-4325-b094-c35009987bea None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:39.920445 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:39.932806 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d262c39d-6a2e-46fc-8f10-37719c0d46ba None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:39.944261 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5047fc-1f24-40f2-8d24-23f32fe67c59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:39.945129 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5047fc-1f24-40f2-8d24-23f32fe67c59 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:39.945415 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5047fc-1f24-40f2-8d24-23f32fe67c59 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:39.945696 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5047fc-1f24-40f2-8d24-23f32fe67c59 None admin] PATH_INFO: `/v3/projects/2240bbdc36ed4e59a7454b287b5891ab` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:39.946911 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5047fc-1f24-40f2-8d24-23f32fe67c59 None admin] RBAC: Authorizing `identity:get_project(project_id=2240bbdc36ed4e59a7454b287b5891ab)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:39.948348 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5047fc-1f24-40f2-8d24-23f32fe67c59 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:39.950029 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 215/444] 10.222.0.46 () {56 vars in 1186 bytes} [Thu Jun 23 15:47:39 2022] GET /identity/v3/projects/2240bbdc36ed4e59a7454b287b5891ab => generated 302 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:47:39.950815 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5047fc-1f24-40f2-8d24-23f32fe67c59 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:39.969637 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1949fc-faa7-43e9-b578-fe910a8484b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:39.970314 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1949fc-faa7-43e9-b578-fe910a8484b3 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:39.970599 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1949fc-faa7-43e9-b578-fe910a8484b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:39.970908 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1949fc-faa7-43e9-b578-fe910a8484b3 None admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:39.971658 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1949fc-faa7-43e9-b578-fe910a8484b3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:47:39.971968 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1949fc-faa7-43e9-b578-fe910a8484b3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:39.973342 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1949fc-faa7-43e9-b578-fe910a8484b3 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:39.982201 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 230/445] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Jun 23 15:47:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:47:39.983022 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1949fc-faa7-43e9-b578-fe910a8484b3 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:40.058464 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5047fc-1f24-40f2-8d24-23f32fe67c59 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:40.081895 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1949fc-faa7-43e9-b578-fe910a8484b3 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:40.083698 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:40.092513 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdc7561-725a-4f8a-9bd0-99ecdaa63370 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:40.093163 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdc7561-725a-4f8a-9bd0-99ecdaa63370 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:40.093447 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdc7561-725a-4f8a-9bd0-99ecdaa63370 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:40.093728 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdc7561-725a-4f8a-9bd0-99ecdaa63370 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:40.121299 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe3548e-7d95-4014-91fb-328e3060e6cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:40.122171 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe3548e-7d95-4014-91fb-328e3060e6cf None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:40.122457 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe3548e-7d95-4014-91fb-328e3060e6cf None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:40.122759 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe3548e-7d95-4014-91fb-328e3060e6cf None admin] PATH_INFO: `/v3/groups/556c754e31a74b81bc55c99683c91208` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:40.124011 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe3548e-7d95-4014-91fb-328e3060e6cf None admin] RBAC: Authorizing `identity:get_group(group_id=556c754e31a74b81bc55c99683c91208)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:40.125496 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe3548e-7d95-4014-91fb-328e3060e6cf None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:40.127179 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 231/446] 10.222.0.46 () {56 vars in 1180 bytes} [Thu Jun 23 15:47:39 2022] GET /identity/v3/groups/556c754e31a74b81bc55c99683c91208 => generated 226 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:47:40.127948 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe3548e-7d95-4014-91fb-328e3060e6cf None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:40.146761 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:40.166200 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fdc7561-725a-4f8a-9bd0-99ecdaa63370 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:40.167651 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fdc7561-725a-4f8a-9bd0-99ecdaa63370 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:40.227079 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe3548e-7d95-4014-91fb-328e3060e6cf None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:40.260433 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7c0010-ec1d-4779-893c-040bf698aabb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:40.261079 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7c0010-ec1d-4779-893c-040bf698aabb None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:40.261358 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7c0010-ec1d-4779-893c-040bf698aabb None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:40.261631 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7c0010-ec1d-4779-893c-040bf698aabb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:40.264407 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 216/447] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Jun 23 15:47:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 15:47:40.265526 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdc7561-725a-4f8a-9bd0-99ecdaa63370 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:40.295616 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:40.329465 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7c0010-ec1d-4779-893c-040bf698aabb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:40.330972 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7c0010-ec1d-4779-893c-040bf698aabb None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:40.348956 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:40.366176 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdc7561-725a-4f8a-9bd0-99ecdaa63370 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:40.400230 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1d6745-8acf-4cad-b7e2-eae30e048c3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:40.401214 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1d6745-8acf-4cad-b7e2-eae30e048c3a None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:40.401501 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1d6745-8acf-4cad-b7e2-eae30e048c3a None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:40.401776 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1d6745-8acf-4cad-b7e2-eae30e048c3a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:40.402602 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1d6745-8acf-4cad-b7e2-eae30e048c3a None admin] RBAC: Adding query filter params (group.id=556c754e31a74b81bc55c99683c91208, role.id=e2a5bbf3200a4d4091d72bc7ff78c34d, scope.project.id=2240bbdc36ed4e59a7454b287b5891ab) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:47:40.402943 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1d6745-8acf-4cad-b7e2-eae30e048c3a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:40.404360 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1d6745-8acf-4cad-b7e2-eae30e048c3a None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:40.410134 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 217/448] 10.222.0.46 () {56 vars in 1376 bytes} [Thu Jun 23 15:47:40 2022] GET /identity/v3/role_assignments?group.id=556c754e31a74b81bc55c99683c91208&scope.project.id=2240bbdc36ed4e59a7454b287b5891ab&role.id=e2a5bbf3200a4d4091d72bc7ff78c34d => generated 611 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:47:40.410941 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1d6745-8acf-4cad-b7e2-eae30e048c3a None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:40.443245 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 232/449] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Jun 23 15:47:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 15:47:40.444935 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7c0010-ec1d-4779-893c-040bf698aabb None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:40.523850 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:40.534684 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1d6745-8acf-4cad-b7e2-eae30e048c3a None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:40.562851 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7c0010-ec1d-4779-893c-040bf698aabb None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:40.579800 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec1a95f-17a1-4185-ab3c-68ca800da02f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:40.580513 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec1a95f-17a1-4185-ab3c-68ca800da02f None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:40.580803 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec1a95f-17a1-4185-ab3c-68ca800da02f None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:40.581083 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec1a95f-17a1-4185-ab3c-68ca800da02f None admin] PATH_INFO: `/v3/projects/2240bbdc36ed4e59a7454b287b5891ab` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:40.582432 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec1a95f-17a1-4185-ab3c-68ca800da02f None admin] RBAC: Authorizing `identity:get_project(project_id=2240bbdc36ed4e59a7454b287b5891ab)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:40.584352 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec1a95f-17a1-4185-ab3c-68ca800da02f None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:40.586885 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:40.588578 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 218/450] 10.222.0.46 () {56 vars in 1186 bytes} [Thu Jun 23 15:47:40 2022] GET /identity/v3/projects/2240bbdc36ed4e59a7454b287b5891ab => generated 302 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:47:40.592739 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec1a95f-17a1-4185-ab3c-68ca800da02f None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:40.612983 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4bea9f-ddb5-444d-8a27-bae9976759a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:40.612983 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4bea9f-ddb5-444d-8a27-bae9976759a9 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:40.612983 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4bea9f-ddb5-444d-8a27-bae9976759a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:40.612983 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4bea9f-ddb5-444d-8a27-bae9976759a9 None admin] PATH_INFO: `/v3/projects/110dee34db4740f989f715e757ee809c` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:40.613437 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4bea9f-ddb5-444d-8a27-bae9976759a9 None admin] RBAC: Authorizing `identity:get_project(project_id=110dee34db4740f989f715e757ee809c)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:40.614442 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4bea9f-ddb5-444d-8a27-bae9976759a9 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:40.616767 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 233/451] 10.222.0.46 () {56 vars in 1186 bytes} [Thu Jun 23 15:47:40 2022] GET /identity/v3/projects/110dee34db4740f989f715e757ee809c => generated 344 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 15:47:40.622058 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4bea9f-ddb5-444d-8a27-bae9976759a9 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:40.732418 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec1a95f-17a1-4185-ab3c-68ca800da02f None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:40.750640 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4bea9f-ddb5-444d-8a27-bae9976759a9 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:40.777542 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9791eff-0d3f-4665-bfad-43c5d2c50b95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:40.778423 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9791eff-0d3f-4665-bfad-43c5d2c50b95 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:40.778727 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9791eff-0d3f-4665-bfad-43c5d2c50b95 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:40.779011 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9791eff-0d3f-4665-bfad-43c5d2c50b95 None admin] PATH_INFO: `/v3/groups/556c754e31a74b81bc55c99683c91208` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:40.780237 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9791eff-0d3f-4665-bfad-43c5d2c50b95 None admin] RBAC: Authorizing `identity:get_group(group_id=556c754e31a74b81bc55c99683c91208)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:40.781716 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9791eff-0d3f-4665-bfad-43c5d2c50b95 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:40.783423 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 219/452] 10.222.0.46 () {56 vars in 1180 bytes} [Thu Jun 23 15:47:40 2022] GET /identity/v3/groups/556c754e31a74b81bc55c99683c91208 => generated 226 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 15:47:40.789435 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9791eff-0d3f-4665-bfad-43c5d2c50b95 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:40.790916 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76988779-a269-47fe-b73c-b61ec573e99e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:40.791713 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76988779-a269-47fe-b73c-b61ec573e99e None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:40.792000 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76988779-a269-47fe-b73c-b61ec573e99e None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:40.792292 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76988779-a269-47fe-b73c-b61ec573e99e None admin] PATH_INFO: `/v3/groups/47834ccd8b0046b2920f4352334f4694` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:40.793485 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76988779-a269-47fe-b73c-b61ec573e99e None admin] RBAC: Authorizing `identity:get_group(group_id=47834ccd8b0046b2920f4352334f4694)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:40.795011 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76988779-a269-47fe-b73c-b61ec573e99e None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:40.796636 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 234/453] 10.222.0.46 () {56 vars in 1180 bytes} [Thu Jun 23 15:47:40 2022] GET /identity/v3/groups/47834ccd8b0046b2920f4352334f4694 => generated 229 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:47:40.802460 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76988779-a269-47fe-b73c-b61ec573e99e None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:40.817287 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:40.859520 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:40.890760 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9791eff-0d3f-4665-bfad-43c5d2c50b95 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:40.904080 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76988779-a269-47fe-b73c-b61ec573e99e None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:40.923711 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833afdc5-931d-47b8-a282-d43f44d2d6fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:40.924450 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833afdc5-931d-47b8-a282-d43f44d2d6fc None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:40.924741 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833afdc5-931d-47b8-a282-d43f44d2d6fc None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:40.925022 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833afdc5-931d-47b8-a282-d43f44d2d6fc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:40.925855 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833afdc5-931d-47b8-a282-d43f44d2d6fc None admin] RBAC: Adding query filter params (group.id=556c754e31a74b81bc55c99683c91208, role.id=d4d557a3ec4e434f8e83499c5fcc7136, scope.project.id=2240bbdc36ed4e59a7454b287b5891ab) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:47:40.926172 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833afdc5-931d-47b8-a282-d43f44d2d6fc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:40.927654 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833afdc5-931d-47b8-a282-d43f44d2d6fc None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:40.934216 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 220/454] 10.222.0.46 () {56 vars in 1376 bytes} [Thu Jun 23 15:47:40 2022] GET /identity/v3/role_assignments?group.id=556c754e31a74b81bc55c99683c91208&scope.project.id=2240bbdc36ed4e59a7454b287b5891ab&role.id=d4d557a3ec4e434f8e83499c5fcc7136 => generated 611 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 15:47:40.939289 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3838a1-492f-4faa-ac64-75c5a112913b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:40.939960 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3838a1-492f-4faa-ac64-75c5a112913b None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:40.940243 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3838a1-492f-4faa-ac64-75c5a112913b None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:40.940521 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3838a1-492f-4faa-ac64-75c5a112913b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:40.941377 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3838a1-492f-4faa-ac64-75c5a112913b None admin] RBAC: Adding query filter params (group.id=47834ccd8b0046b2920f4352334f4694, role.id=812d3c567c52469cb2b35709fbdc857c, scope.project.id=110dee34db4740f989f715e757ee809c) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:47:40.941826 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3838a1-492f-4faa-ac64-75c5a112913b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:40.943237 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3838a1-492f-4faa-ac64-75c5a112913b None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:40.950684 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 235/455] 10.222.0.46 () {56 vars in 1376 bytes} [Thu Jun 23 15:47:40 2022] GET /identity/v3/role_assignments?group.id=47834ccd8b0046b2920f4352334f4694&scope.project.id=110dee34db4740f989f715e757ee809c&role.id=812d3c567c52469cb2b35709fbdc857c => generated 611 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 15:47:43.616902 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37cacb6-6e07-44bf-843a-1281af20dc03 None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:43.617752 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37cacb6-6e07-44bf-843a-1281af20dc03 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:43.618198 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37cacb6-6e07-44bf-843a-1281af20dc03 None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:43.619973 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 221/456] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jun 23 15:47:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 15:47:43.625602 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0297adf-c678-4054-b99a-9184c5ab70e6 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:43.626370 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0297adf-c678-4054-b99a-9184c5ab70e6 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:43.626818 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0297adf-c678-4054-b99a-9184c5ab70e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:43.655708 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-a0297adf-c678-4054-b99a-9184c5ab70e6 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:47:43.657169 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-a0297adf-c678-4054-b99a-9184c5ab70e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:47:43.744688 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 236/457] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jun 23 15:47:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 15:47:43.784506 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37cacb6-6e07-44bf-843a-1281af20dc03 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:43.873765 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1114ed7-59e6-4c6f-b3e3-9ff4f30f10ba None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:43.874107 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1114ed7-59e6-4c6f-b3e3-9ff4f30f10ba None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:43.874329 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1114ed7-59e6-4c6f-b3e3-9ff4f30f10ba None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:43.875734 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 237/458] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jun 23 15:47:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 15:47:43.886407 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1e79b8-053c-4f09-8b6f-01569b542775 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:43.887023 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1e79b8-053c-4f09-8b6f-01569b542775 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:43.887502 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1e79b8-053c-4f09-8b6f-01569b542775 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:43.913198 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-9e1e79b8-053c-4f09-8b6f-01569b542775 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:47:43.915028 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:43.915865 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-9e1e79b8-053c-4f09-8b6f-01569b542775 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:47:43.932737 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37cacb6-6e07-44bf-843a-1281af20dc03 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:43.982151 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a73d510-3622-45aa-b572-2f6b93e654ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:43.983455 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a73d510-3622-45aa-b572-2f6b93e654ab None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:43.983765 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a73d510-3622-45aa-b572-2f6b93e654ab None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:43.984045 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a73d510-3622-45aa-b572-2f6b93e654ab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:43.985446 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:43.999400 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a73d510-3622-45aa-b572-2f6b93e654ab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:44.001424 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a73d510-3622-45aa-b572-2f6b93e654ab None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:44.013729 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 238/459] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jun 23 15:47:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 15:47:44.015485 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b18da64-13ec-4a4a-be8e-51bff06135aa None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:44.015975 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b18da64-13ec-4a4a-be8e-51bff06135aa None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:44.016464 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b18da64-13ec-4a4a-be8e-51bff06135aa None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:44.017948 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 239/460] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jun 23 15:47:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 15:47:44.023553 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a101c77e-5708-453c-a342-85b20884c887 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:44.024133 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a101c77e-5708-453c-a342-85b20884c887 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:44.024601 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a101c77e-5708-453c-a342-85b20884c887 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:44.033239 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-3a73d510-3622-45aa-b572-2f6b93e654ab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 15:47:44.033239 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:47:44.033239 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:47:44.033239 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:47:44.033239 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:47:44.033239 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:47:44.033239 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:47:44.033239 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:47:44.033239 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:47:44.033239 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:47:44.033239 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:47:44.033239 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:47:44.033239 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 15:47:44.033239 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 15:47:44.033239 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 15:47:44.034301 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 15:47:44.034301 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:47:44.034301 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:47:44.034301 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:47:44.034301 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:47:44.034301 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:47:44.034301 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:47:44.034301 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:47:44.034301 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Jun 23 15:47:44.034301 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:47:44.034301 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:47:44.034301 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:47:44.034301 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:47:44.034301 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:47:44.034301 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:47:44.034301 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:47:44.034301 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:47:44.035368 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 15:47:44.035368 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 15:47:44.035368 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 15:47:44.035368 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Jun 23 15:47:44.037230 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 222/461] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Jun 23 15:47:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:47:44.042947 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a73d510-3622-45aa-b572-2f6b93e654ab None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:44.049646 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-a101c77e-5708-453c-a342-85b20884c887 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:47:44.051194 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-a101c77e-5708-453c-a342-85b20884c887 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:47:44.150918 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 240/462] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jun 23 15:47:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 15:47:44.151679 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a101c77e-5708-453c-a342-85b20884c887 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:44.179430 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a73d510-3622-45aa-b572-2f6b93e654ab None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:44.228054 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd24ffdd-9655-45a1-93b6-3e5b4366a724 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:44.229375 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd24ffdd-9655-45a1-93b6-3e5b4366a724 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:44.230054 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd24ffdd-9655-45a1-93b6-3e5b4366a724 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:44.230472 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd24ffdd-9655-45a1-93b6-3e5b4366a724 None admin] PATH_INFO: `/v3/domains` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:44.231381 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd24ffdd-9655-45a1-93b6-3e5b4366a724 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:47:44.231827 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd24ffdd-9655-45a1-93b6-3e5b4366a724 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:44.234198 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd24ffdd-9655-45a1-93b6-3e5b4366a724 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:44.251186 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 223/463] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Jun 23 15:47:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:47:44.252308 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd24ffdd-9655-45a1-93b6-3e5b4366a724 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:44.256761 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:44.275792 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a101c77e-5708-453c-a342-85b20884c887 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:44.287155 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:44.317509 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be4baca-4a95-42d7-8ccf-c794bb46d5e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:44.318512 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be4baca-4a95-42d7-8ccf-c794bb46d5e5 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:44.318891 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be4baca-4a95-42d7-8ccf-c794bb46d5e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:44.319302 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be4baca-4a95-42d7-8ccf-c794bb46d5e5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:44.331174 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be4baca-4a95-42d7-8ccf-c794bb46d5e5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:44.333014 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be4baca-4a95-42d7-8ccf-c794bb46d5e5 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:44.353521 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-7be4baca-4a95-42d7-8ccf-c794bb46d5e5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 15:47:44.353521 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:47:44.353521 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:47:44.353521 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:47:44.353521 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:47:44.353521 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:47:44.353521 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:47:44.353521 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:47:44.353521 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:47:44.353521 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:47:44.353521 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:47:44.353521 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:47:44.353521 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 15:47:44.353521 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 15:47:44.353521 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 15:47:44.354734 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 15:47:44.354734 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:47:44.354734 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:47:44.354734 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:47:44.354734 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:47:44.354734 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:47:44.354734 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:47:44.354734 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:47:44.354734 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Jun 23 15:47:44.354734 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:47:44.354734 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:47:44.354734 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:47:44.354734 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:47:44.354734 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:47:44.354734 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:47:44.354734 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:47:44.354734 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:47:44.355907 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 15:47:44.355907 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 15:47:44.355907 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 15:47:44.355907 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Jun 23 15:47:44.358748 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 241/464] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Jun 23 15:47:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:47:44.358748 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be4baca-4a95-42d7-8ccf-c794bb46d5e5 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:44.395741 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd24ffdd-9655-45a1-93b6-3e5b4366a724 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:44.431907 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c320f60-f9b1-42ce-adf1-73e5efbcb863 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:44.432816 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c320f60-f9b1-42ce-adf1-73e5efbcb863 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:44.433211 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c320f60-f9b1-42ce-adf1-73e5efbcb863 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:44.433609 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c320f60-f9b1-42ce-adf1-73e5efbcb863 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:44.445125 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c320f60-f9b1-42ce-adf1-73e5efbcb863 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:44.446731 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c320f60-f9b1-42ce-adf1-73e5efbcb863 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:44.464705 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-4c320f60-f9b1-42ce-adf1-73e5efbcb863 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 15:47:44.464705 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:47:44.464705 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:47:44.464705 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:47:44.464705 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:47:44.464705 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:47:44.464705 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:47:44.464705 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:47:44.464705 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:47:44.464705 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:47:44.464705 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:47:44.464705 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:47:44.464705 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 15:47:44.464705 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 15:47:44.464705 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 15:47:44.465802 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 15:47:44.465802 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:47:44.465802 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:47:44.465802 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:47:44.465802 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:47:44.465802 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:47:44.465802 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:47:44.465802 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:47:44.465802 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Jun 23 15:47:44.465802 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:47:44.465802 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:47:44.465802 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:47:44.465802 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:47:44.465802 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:47:44.465802 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:47:44.465802 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:47:44.465802 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:47:44.466668 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 15:47:44.466668 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 15:47:44.466668 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 15:47:44.466668 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Jun 23 15:47:44.470961 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 224/465] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Jun 23 15:47:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:47:44.471839 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c320f60-f9b1-42ce-adf1-73e5efbcb863 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:44.478268 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be4baca-4a95-42d7-8ccf-c794bb46d5e5 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:44.522666 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d4f19a-3d43-4715-a66c-817fe298feca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:44.523642 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d4f19a-3d43-4715-a66c-817fe298feca None admin] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:44.523930 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d4f19a-3d43-4715-a66c-817fe298feca None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:44.524206 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d4f19a-3d43-4715-a66c-817fe298feca None admin] PATH_INFO: `/v3/users` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:44.525081 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d4f19a-3d43-4715-a66c-817fe298feca None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:44.527123 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d4f19a-3d43-4715-a66c-817fe298feca None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:44.532711 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:44.557515 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 242/466] 10.222.0.46 () {60 vars in 1132 bytes} [Thu Jun 23 15:47:44 2022] POST /identity/v3/users => generated 246 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 15:47:44.558492 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d4f19a-3d43-4715-a66c-817fe298feca None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:44.599150 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:44.626567 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c320f60-f9b1-42ce-adf1-73e5efbcb863 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:44.668759 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972de63c-8ca0-421f-833a-1e898f60bf43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:44.669503 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972de63c-8ca0-421f-833a-1e898f60bf43 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:44.669794 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972de63c-8ca0-421f-833a-1e898f60bf43 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:44.670202 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972de63c-8ca0-421f-833a-1e898f60bf43 None admin] PATH_INFO: `/v3/domains` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:44.670839 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972de63c-8ca0-421f-833a-1e898f60bf43 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:47:44.671153 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972de63c-8ca0-421f-833a-1e898f60bf43 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:44.672568 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972de63c-8ca0-421f-833a-1e898f60bf43 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:44.691027 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 225/467] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Jun 23 15:47:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:47:44.691027 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af75d98a-20df-4b65-9757-870dfd853acb None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:44.691027 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af75d98a-20df-4b65-9757-870dfd853acb None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:44.691346 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af75d98a-20df-4b65-9757-870dfd853acb None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:44.693848 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 226/468] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jun 23 15:47:44 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 15:47:44.696314 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add4a766-458e-4cee-9fa6-736357173dc0 None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:44.697327 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add4a766-458e-4cee-9fa6-736357173dc0 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:44.697327 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add4a766-458e-4cee-9fa6-736357173dc0 None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:44.702745 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 227/469] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jun 23 15:47:44 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 15:47:44.702745 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add4a766-458e-4cee-9fa6-736357173dc0 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:44.716654 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d4f19a-3d43-4715-a66c-817fe298feca None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:44.759376 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fb54e0-4d86-428b-bea6-d19dbb787092 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:44.759881 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fb54e0-4d86-428b-bea6-d19dbb787092 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:44.760742 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fb54e0-4d86-428b-bea6-d19dbb787092 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:44.760742 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fb54e0-4d86-428b-bea6-d19dbb787092 None admin] PATH_INFO: `/v3/domains` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:44.761148 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fb54e0-4d86-428b-bea6-d19dbb787092 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:47:44.761449 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fb54e0-4d86-428b-bea6-d19dbb787092 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:44.763478 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fb54e0-4d86-428b-bea6-d19dbb787092 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:44.778653 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 243/470] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Jun 23 15:47:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:47:44.779810 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1eec31-e9e1-4cd0-b695-5630b071eae0 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:44.780093 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1eec31-e9e1-4cd0-b695-5630b071eae0 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:44.780573 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1eec31-e9e1-4cd0-b695-5630b071eae0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:44.800008 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-cc1eec31-e9e1-4cd0-b695-5630b071eae0 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:47:44.801436 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-cc1eec31-e9e1-4cd0-b695-5630b071eae0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:47:44.820941 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add4a766-458e-4cee-9fa6-736357173dc0 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:44.840238 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:44.861964 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce23dfea-7ce4-41a5-be79-ffcea5b92cec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:44.861964 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce23dfea-7ce4-41a5-be79-ffcea5b92cec None admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:44.862851 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce23dfea-7ce4-41a5-be79-ffcea5b92cec None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:44.862977 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce23dfea-7ce4-41a5-be79-ffcea5b92cec None admin] PATH_INFO: `/v3/users` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:44.863872 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce23dfea-7ce4-41a5-be79-ffcea5b92cec None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:44.865344 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce23dfea-7ce4-41a5-be79-ffcea5b92cec None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:44.877653 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 244/471] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jun 23 15:47:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 15:47:44.879002 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d8bc4e-1101-418b-aa4f-f9ca05e72c44 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:44.879272 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d8bc4e-1101-418b-aa4f-f9ca05e72c44 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:44.879567 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d8bc4e-1101-418b-aa4f-f9ca05e72c44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:44.891965 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:44.895982 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 228/472] 10.222.0.46 () {60 vars in 1132 bytes} [Thu Jun 23 15:47:44 2022] POST /identity/v3/users => generated 248 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 15:47:44.897110 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce23dfea-7ce4-41a5-be79-ffcea5b92cec None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:44.899593 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-d9d8bc4e-1101-418b-aa4f-f9ca05e72c44 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:47:44.901253 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-d9d8bc4e-1101-418b-aa4f-f9ca05e72c44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:47:44.984625 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 245/473] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jun 23 15:47:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 15:47:44.985023 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d8bc4e-1101-418b-aa4f-f9ca05e72c44 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:45.024514 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce23dfea-7ce4-41a5-be79-ffcea5b92cec None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:45.068982 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e466a2aa-620e-4cd1-952d-908f31c5c5de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:45.070026 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e466a2aa-620e-4cd1-952d-908f31c5c5de None admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:45.070459 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e466a2aa-620e-4cd1-952d-908f31c5c5de None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:45.070900 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e466a2aa-620e-4cd1-952d-908f31c5c5de None admin] PATH_INFO: `/v3/users` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:45.071772 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e466a2aa-620e-4cd1-952d-908f31c5c5de None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:45.073366 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e466a2aa-620e-4cd1-952d-908f31c5c5de None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:45.096578 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 229/474] 10.222.0.46 () {60 vars in 1132 bytes} [Thu Jun 23 15:47:44 2022] POST /identity/v3/users => generated 248 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 15:47:45.097571 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e466a2aa-620e-4cd1-952d-908f31c5c5de None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:45.119426 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d8bc4e-1101-418b-aa4f-f9ca05e72c44 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:45.170257 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a72c5b9-c539-4d58-984e-64a646fed7e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:45.170257 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a72c5b9-c539-4d58-984e-64a646fed7e5 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:45.170537 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a72c5b9-c539-4d58-984e-64a646fed7e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:45.170739 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a72c5b9-c539-4d58-984e-64a646fed7e5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:45.171041 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:45.174757 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:45.187939 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a72c5b9-c539-4d58-984e-64a646fed7e5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:45.189852 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a72c5b9-c539-4d58-984e-64a646fed7e5 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:45.215423 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-9a72c5b9-c539-4d58-984e-64a646fed7e5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 15:47:45.215423 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:47:45.215423 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:47:45.215423 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:47:45.215423 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:47:45.215423 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:47:45.215423 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:47:45.215423 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:47:45.215423 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:47:45.215423 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:47:45.215423 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:47:45.215423 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:47:45.215423 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 15:47:45.215423 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 15:47:45.215423 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 15:47:45.216390 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 15:47:45.216390 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:47:45.216390 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:47:45.216390 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:47:45.216390 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:47:45.216390 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:47:45.216390 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:47:45.216390 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:47:45.216390 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Jun 23 15:47:45.216390 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:47:45.216390 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:47:45.216390 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:47:45.216390 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:47:45.216390 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:47:45.216390 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:47:45.216390 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:47:45.216390 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:47:45.217211 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 15:47:45.217211 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 15:47:45.217211 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 15:47:45.217211 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Jun 23 15:47:45.217211 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 246/475] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Jun 23 15:47:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:47:45.223649 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a72c5b9-c539-4d58-984e-64a646fed7e5 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:45.237570 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e466a2aa-620e-4cd1-952d-908f31c5c5de None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:45.282052 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739d59f1-8c23-4c3f-a40f-8ef25caaef4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:45.282433 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739d59f1-8c23-4c3f-a40f-8ef25caaef4e None admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:45.283869 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739d59f1-8c23-4c3f-a40f-8ef25caaef4e None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:45.283869 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739d59f1-8c23-4c3f-a40f-8ef25caaef4e None admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:45.284893 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739d59f1-8c23-4c3f-a40f-8ef25caaef4e None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:45.285966 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739d59f1-8c23-4c3f-a40f-8ef25caaef4e None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:45.295895 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.sql.core [None req-739d59f1-8c23-4c3f-a40f-8ef25caaef4e None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 23 15:47:45.295895 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jun 23 15:47:45.295895 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [parameters: {'id': 'd4dd66a9dd004a0b8fa6f9c6a8a3e152', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jun 23 15:47:45.295895 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80154) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 23 15:47:45.313477 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-739d59f1-8c23-4c3f-a40f-8ef25caaef4e None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jun 23 15:47:45.313477 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:47:45.313477 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 23 15:47:45.313477 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 23 15:47:45.313477 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 23 15:47:45.313477 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 23 15:47:45.313477 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 23 15:47:45.313477 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application result = self._query(query) Jun 23 15:47:45.313477 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 23 15:47:45.313477 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application conn.query(q) Jun 23 15:47:45.313477 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 23 15:47:45.313477 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 23 15:47:45.313477 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 23 15:47:45.313477 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application result.read() Jun 23 15:47:45.313477 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 23 15:47:45.314514 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 23 15:47:45.314514 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 23 15:47:45.314514 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 23 15:47:45.314514 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 23 15:47:45.314514 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 23 15:47:45.314514 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 23 15:47:45.314514 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 23 15:47:45.314514 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 23 15:47:45.314514 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Jun 23 15:47:45.314514 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 23 15:47:45.314514 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Jun 23 15:47:45.314514 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:47:45.314514 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 23 15:47:45.314514 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 23 15:47:45.314514 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jun 23 15:47:45.314514 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return ref.to_dict() Jun 23 15:47:45.314514 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 23 15:47:45.314514 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application next(self.gen) Jun 23 15:47:45.315537 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 23 15:47:45.315537 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application yield resource Jun 23 15:47:45.315537 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 23 15:47:45.315537 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application next(self.gen) Jun 23 15:47:45.315537 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 23 15:47:45.315537 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application self.session.rollback() Jun 23 15:47:45.315537 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 23 15:47:45.315537 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application self.force_reraise() Jun 23 15:47:45.315537 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 23 15:47:45.315537 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise self.value Jun 23 15:47:45.315537 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 23 15:47:45.315537 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 23 15:47:45.315537 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 23 15:47:45.315537 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application session.commit() Jun 23 15:47:45.315537 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 23 15:47:45.315537 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 23 15:47:45.315537 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 23 15:47:45.316382 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application self._prepare_impl() Jun 23 15:47:45.316382 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 23 15:47:45.316382 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application self.session.flush() Jun 23 15:47:45.316382 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 23 15:47:45.316382 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application self._flush(objects) Jun 23 15:47:45.316382 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 23 15:47:45.316382 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 23 15:47:45.316382 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 23 15:47:45.316382 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application compat.raise_( Jun 23 15:47:45.316382 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 23 15:47:45.316382 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception Jun 23 15:47:45.316382 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 23 15:47:45.316382 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application flush_context.execute() Jun 23 15:47:45.316382 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 23 15:47:45.316382 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rec.execute(self) Jun 23 15:47:45.316382 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 23 15:47:45.316382 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 23 15:47:45.317243 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 23 15:47:45.317243 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 23 15:47:45.317243 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jun 23 15:47:45.317243 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application c = connection._execute_20( Jun 23 15:47:45.317243 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 23 15:47:45.317243 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 23 15:47:45.317243 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 23 15:47:45.317243 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 23 15:47:45.317243 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 23 15:47:45.317243 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 23 15:47:45.317243 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 23 15:47:45.317243 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 23 15:47:45.317243 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 23 15:47:45.317243 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 23 15:47:45.317243 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 23 15:47:45.317243 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception Jun 23 15:47:45.318062 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 23 15:47:45.318062 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 23 15:47:45.318062 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 23 15:47:45.318062 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 23 15:47:45.318062 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 23 15:47:45.318062 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application result = self._query(query) Jun 23 15:47:45.318062 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 23 15:47:45.318062 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application conn.query(q) Jun 23 15:47:45.318062 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 23 15:47:45.318062 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 23 15:47:45.318062 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 23 15:47:45.318062 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application result.read() Jun 23 15:47:45.318062 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 23 15:47:45.318062 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 23 15:47:45.318062 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 23 15:47:45.318062 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 23 15:47:45.320085 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 23 15:47:45.320085 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 23 15:47:45.320085 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 23 15:47:45.320085 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 23 15:47:45.320085 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 23 15:47:45.320085 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jun 23 15:47:45.320085 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [parameters: {'id': 'd4dd66a9dd004a0b8fa6f9c6a8a3e152', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jun 23 15:47:45.320085 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 23 15:47:45.320085 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Jun 23 15:47:45.320085 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 15:47:45.320085 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Jun 23 15:47:45.320085 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:47:45.320085 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:47:45.320085 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:47:45.320085 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:47:45.320085 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:47:45.322124 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:47:45.322124 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:47:45.322124 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:47:45.322124 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:47:45.322124 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:47:45.322124 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:47:45.322124 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jun 23 15:47:45.322124 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jun 23 15:47:45.322124 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:47:45.322124 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:47:45.322124 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jun 23 15:47:45.322124 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jun 23 15:47:45.322124 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 23 15:47:45.322124 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 23 15:47:45.322124 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jun 23 15:47:45.322124 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Jun 23 15:47:45.329756 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 230/476] 10.222.0.46 () {60 vars in 1131 bytes} [Thu Jun 23 15:47:45 2022] POST /identity/v3/roles => generated 144 bytes in 218 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:47:45.329756 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739d59f1-8c23-4c3f-a40f-8ef25caaef4e None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:45.357798 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a72c5b9-c539-4d58-984e-64a646fed7e5 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:45.407019 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79fd341-5e74-4246-9833-a53e60e7fcd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:45.407704 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79fd341-5e74-4246-9833-a53e60e7fcd7 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:45.407986 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79fd341-5e74-4246-9833-a53e60e7fcd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:45.408263 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79fd341-5e74-4246-9833-a53e60e7fcd7 None admin] PATH_INFO: `/v3/domains` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:45.409005 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79fd341-5e74-4246-9833-a53e60e7fcd7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:47:45.409604 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79fd341-5e74-4246-9833-a53e60e7fcd7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:45.411044 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79fd341-5e74-4246-9833-a53e60e7fcd7 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:45.425521 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 247/477] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Jun 23 15:47:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 15:47:45.431213 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79fd341-5e74-4246-9833-a53e60e7fcd7 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:45.441515 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:45.452323 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739d59f1-8c23-4c3f-a40f-8ef25caaef4e None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:45.487692 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:45.497330 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51afeda-53ee-47ec-acb3-2b5c994ac8d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:45.498827 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51afeda-53ee-47ec-acb3-2b5c994ac8d9 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:45.499217 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51afeda-53ee-47ec-acb3-2b5c994ac8d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:45.500259 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51afeda-53ee-47ec-acb3-2b5c994ac8d9 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:45.515117 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51afeda-53ee-47ec-acb3-2b5c994ac8d9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 15:47:45.526193 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-c51afeda-53ee-47ec-acb3-2b5c994ac8d9 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 15:47:45.526193 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:47:45.526193 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:47:45.526193 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:47:45.526193 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:47:45.526193 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:47:45.526193 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:47:45.526193 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:47:45.526193 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:47:45.526193 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:47:45.526193 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:47:45.526193 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:47:45.526193 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 15:47:45.526193 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 15:47:45.526193 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 15:47:45.526193 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 15:47:45.527160 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 15:47:45.527160 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 15:47:45.527160 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 15:47:45.527160 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 15:47:45.527160 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:47:45.527160 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:47:45.527160 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:47:45.527160 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:47:45.527160 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:47:45.527160 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:47:45.527160 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:47:45.527160 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Jun 23 15:47:45.527160 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:47:45.527160 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:47:45.527160 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:47:45.527160 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:47:45.527947 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:47:45.527947 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:47:45.527947 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:47:45.527947 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:47:45.527947 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 15:47:45.527947 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 15:47:45.527947 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 15:47:45.527947 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 15:47:45.527947 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 15:47:45.527947 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 15:47:45.527947 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 23 15:47:45.527947 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Jun 23 15:47:45.530836 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 231/478] 10.222.0.46 () {56 vars in 1114 bytes} [Thu Jun 23 15:47:45 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 15:47:45.534776 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51afeda-53ee-47ec-acb3-2b5c994ac8d9 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:45.567455 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79fd341-5e74-4246-9833-a53e60e7fcd7 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:45.615959 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad2013f-ef75-4372-96e3-699caa0adb4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:45.617030 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad2013f-ef75-4372-96e3-699caa0adb4b None admin] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:45.617751 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad2013f-ef75-4372-96e3-699caa0adb4b None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:45.618222 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad2013f-ef75-4372-96e3-699caa0adb4b None admin] PATH_INFO: `/v3/users` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:45.619723 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad2013f-ef75-4372-96e3-699caa0adb4b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:45.622282 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad2013f-ef75-4372-96e3-699caa0adb4b None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:45.656154 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 248/479] 10.222.0.46 () {60 vars in 1132 bytes} [Thu Jun 23 15:47:45 2022] POST /identity/v3/users => generated 249 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 15:47:45.698131 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51afeda-53ee-47ec-acb3-2b5c994ac8d9 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:45.749282 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997660f7-a1bd-4fb8-8ab2-7ebaebfe678b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:45.749887 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997660f7-a1bd-4fb8-8ab2-7ebaebfe678b None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:45.750168 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997660f7-a1bd-4fb8-8ab2-7ebaebfe678b None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:45.750442 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997660f7-a1bd-4fb8-8ab2-7ebaebfe678b None admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:45.751232 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997660f7-a1bd-4fb8-8ab2-7ebaebfe678b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:47:45.751538 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997660f7-a1bd-4fb8-8ab2-7ebaebfe678b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:45.753016 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997660f7-a1bd-4fb8-8ab2-7ebaebfe678b None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:45.767375 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 232/480] 10.222.0.46 () {56 vars in 1111 bytes} [Thu Jun 23 15:47:45 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 15:47:47.007804 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cf42cf-7904-446d-9c85-eb5d9f83882b None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:47.008317 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cf42cf-7904-446d-9c85-eb5d9f83882b None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:47.008417 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cf42cf-7904-446d-9c85-eb5d9f83882b None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:47.009945 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 249/481] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jun 23 15:47:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 15:47:47.015395 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e186e21-3fc9-498e-a2e4-832fbc31fcd2 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:47.015805 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e186e21-3fc9-498e-a2e4-832fbc31fcd2 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:47.015972 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e186e21-3fc9-498e-a2e4-832fbc31fcd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:47.033833 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-7e186e21-3fc9-498e-a2e4-832fbc31fcd2 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:47:47.035056 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-7e186e21-3fc9-498e-a2e4-832fbc31fcd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:47:47.045117 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:47.104688 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 233/482] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jun 23 15:47:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 15:47:47.127675 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cf42cf-7904-446d-9c85-eb5d9f83882b None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:47.242851 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cf42cf-7904-446d-9c85-eb5d9f83882b None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:47.245283 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:47.281271 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f5c504-9248-429e-8de4-831ff201fedf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:47.281987 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f5c504-9248-429e-8de4-831ff201fedf None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:47.282269 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f5c504-9248-429e-8de4-831ff201fedf None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:47.282539 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f5c504-9248-429e-8de4-831ff201fedf None admin] PATH_INFO: `/v3/roles/service` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:47.293263 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f5c504-9248-429e-8de4-831ff201fedf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 15:47:47.300928 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-e0f5c504-9248-429e-8de4-831ff201fedf None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 23 15:47:47.300928 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:47:47.300928 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:47:47.300928 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:47:47.300928 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:47:47.300928 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:47:47.300928 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:47:47.300928 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:47:47.300928 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:47:47.300928 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:47:47.300928 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:47:47.300928 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:47:47.300928 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 15:47:47.300928 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 15:47:47.300928 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 15:47:47.300928 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 15:47:47.301886 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 15:47:47.301886 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 15:47:47.301886 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 15:47:47.301886 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 15:47:47.301886 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:47:47.301886 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:47:47.301886 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:47:47.301886 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:47:47.301886 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:47:47.301886 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:47:47.301886 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:47:47.301886 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Jun 23 15:47:47.301886 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:47:47.301886 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:47:47.301886 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:47:47.301886 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:47:47.303107 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:47:47.303107 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:47:47.303107 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:47:47.303107 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:47:47.303107 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 15:47:47.303107 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 15:47:47.303107 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 15:47:47.303107 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 15:47:47.303107 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 15:47:47.303107 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 15:47:47.303107 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 23 15:47:47.303107 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Jun 23 15:47:47.303107 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 250/483] 10.222.0.46 () {56 vars in 1102 bytes} [Thu Jun 23 15:47:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:47:47.308350 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e186e21-3fc9-498e-a2e4-832fbc31fcd2 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:47.440832 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e186e21-3fc9-498e-a2e4-832fbc31fcd2 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:47.486838 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca1129c-2174-4a99-983d-43d0c03c39c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:47.487526 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca1129c-2174-4a99-983d-43d0c03c39c4 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:47.487961 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca1129c-2174-4a99-983d-43d0c03c39c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:47.488433 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca1129c-2174-4a99-983d-43d0c03c39c4 None admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:47.489325 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca1129c-2174-4a99-983d-43d0c03c39c4 None admin] RBAC: Adding query filter params (name=service) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:47:47.489635 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca1129c-2174-4a99-983d-43d0c03c39c4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:47.491991 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca1129c-2174-4a99-983d-43d0c03c39c4 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:47.501074 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 234/484] 10.222.0.46 () {56 vars in 1103 bytes} [Thu Jun 23 15:47:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:47:47.506277 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f5c504-9248-429e-8de4-831ff201fedf None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:47.638264 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f5c504-9248-429e-8de4-831ff201fedf None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:47.674280 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c3a92f-681d-4aa7-bacd-99f1e3fc40b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:47.675103 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c3a92f-681d-4aa7-bacd-99f1e3fc40b4 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:47.675380 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c3a92f-681d-4aa7-bacd-99f1e3fc40b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:47.675648 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c3a92f-681d-4aa7-bacd-99f1e3fc40b4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:47.686846 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c3a92f-681d-4aa7-bacd-99f1e3fc40b4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:47.688330 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c3a92f-681d-4aa7-bacd-99f1e3fc40b4 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:47.705955 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-a2c3a92f-681d-4aa7-bacd-99f1e3fc40b4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 15:47:47.705955 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:47:47.705955 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:47:47.705955 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:47:47.705955 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:47:47.705955 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:47:47.705955 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:47:47.705955 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:47:47.705955 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:47:47.705955 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:47:47.705955 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:47:47.705955 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:47:47.705955 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 15:47:47.705955 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 15:47:47.705955 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 15:47:47.706882 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 15:47:47.706882 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:47:47.706882 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:47:47.706882 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:47:47.706882 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:47:47.706882 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:47:47.706882 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:47:47.706882 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:47:47.706882 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Jun 23 15:47:47.706882 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:47:47.706882 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:47:47.706882 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:47:47.706882 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:47:47.706882 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:47:47.706882 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:47:47.706882 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:47:47.706882 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:47:47.708142 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 15:47:47.708142 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 15:47:47.708142 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 15:47:47.708142 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Jun 23 15:47:47.708142 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 251/485] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Jun 23 15:47:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:47:47.713606 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca1129c-2174-4a99-983d-43d0c03c39c4 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:47.733045 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:47.815960 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3161c4-cb83-478f-bfb5-5f14370bf40f None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:47.816280 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3161c4-cb83-478f-bfb5-5f14370bf40f None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:47.816510 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3161c4-cb83-478f-bfb5-5f14370bf40f None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:47.818141 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 252/486] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jun 23 15:47:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 15:47:47.828728 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5b29f1-3c87-40f1-a8a3-b06a8af63088 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:47.828997 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5b29f1-3c87-40f1-a8a3-b06a8af63088 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:47.829223 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5b29f1-3c87-40f1-a8a3-b06a8af63088 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:47.853852 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca1129c-2174-4a99-983d-43d0c03c39c4 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:47.863679 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-0b5b29f1-3c87-40f1-a8a3-b06a8af63088 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:47:47.864905 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-0b5b29f1-3c87-40f1-a8a3-b06a8af63088 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:47:47.888575 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:47.893515 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef74e1a9-d5bd-42f0-9abf-9f06559e1429 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:47.894327 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef74e1a9-d5bd-42f0-9abf-9f06559e1429 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:47.894612 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef74e1a9-d5bd-42f0-9abf-9f06559e1429 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:47.894906 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef74e1a9-d5bd-42f0-9abf-9f06559e1429 None admin] PATH_INFO: `/v3/domains` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:47.895639 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef74e1a9-d5bd-42f0-9abf-9f06559e1429 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:47:47.895946 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef74e1a9-d5bd-42f0-9abf-9f06559e1429 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:47.897894 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef74e1a9-d5bd-42f0-9abf-9f06559e1429 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:47.912465 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 235/487] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Jun 23 15:47:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:47:47.917782 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef74e1a9-d5bd-42f0-9abf-9f06559e1429 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:47.992133 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:48.033607 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef74e1a9-d5bd-42f0-9abf-9f06559e1429 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:48.062842 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 253/488] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jun 23 15:47:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 15:47:48.064341 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4896f2-6bf2-4dbb-8bb8-b54b9e4d127e None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:48.064635 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4896f2-6bf2-4dbb-8bb8-b54b9e4d127e None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:48.064917 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4896f2-6bf2-4dbb-8bb8-b54b9e4d127e None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:48.066280 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 254/489] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jun 23 15:47:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 15:47:48.071676 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f94bb2e-e1d8-40a7-b593-3d589d021d70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:48.072064 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5362ed83-cd6c-4001-bc86-2bd5d7e5296b None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:48.072358 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5362ed83-cd6c-4001-bc86-2bd5d7e5296b None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:48.072520 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f94bb2e-e1d8-40a7-b593-3d589d021d70 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:48.072678 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5362ed83-cd6c-4001-bc86-2bd5d7e5296b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:48.072820 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f94bb2e-e1d8-40a7-b593-3d589d021d70 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:48.073002 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f94bb2e-e1d8-40a7-b593-3d589d021d70 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:48.082077 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f94bb2e-e1d8-40a7-b593-3d589d021d70 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:48.083547 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f94bb2e-e1d8-40a7-b593-3d589d021d70 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:48.091249 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-5362ed83-cd6c-4001-bc86-2bd5d7e5296b None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:47:48.092448 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-5362ed83-cd6c-4001-bc86-2bd5d7e5296b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:47:48.103537 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-7f94bb2e-e1d8-40a7-b593-3d589d021d70 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 23 15:47:48.103537 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:47:48.103537 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:47:48.103537 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:47:48.103537 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:47:48.103537 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:47:48.103537 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:47:48.103537 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:47:48.103537 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:47:48.103537 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:47:48.103537 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:47:48.103537 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:47:48.103537 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 15:47:48.103537 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 15:47:48.103537 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 15:47:48.103537 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 15:47:48.105187 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:47:48.105187 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:47:48.105187 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 15:47:48.105187 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 15:47:48.105187 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 15:47:48.105187 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 15:47:48.105187 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:47:48.105187 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:47:48.105187 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:47:48.105187 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:47:48.105187 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:47:48.105187 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Jun 23 15:47:48.105187 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:47:48.105187 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:47:48.105187 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:47:48.105187 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:47:48.105187 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:47:48.105187 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:47:48.106866 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:47:48.106866 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:47:48.106866 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 15:47:48.106866 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 15:47:48.106866 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 15:47:48.106866 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 15:47:48.106866 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 15:47:48.106866 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 15:47:48.106866 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 23 15:47:48.106866 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Jun 23 15:47:48.106866 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 236/490] 10.222.0.46 () {56 vars in 1093 bytes} [Thu Jun 23 15:47:47 2022] GET /identity/v3/users/nova => generated 82 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 15:47:48.110951 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f94bb2e-e1d8-40a7-b593-3d589d021d70 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:48.171279 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 255/491] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jun 23 15:47:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 15:47:48.179885 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5362ed83-cd6c-4001-bc86-2bd5d7e5296b None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:48.235351 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f94bb2e-e1d8-40a7-b593-3d589d021d70 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:48.268620 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:48.278814 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5b1c1a-e452-4b68-a358-3a4ed3f78c2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:48.279608 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5b1c1a-e452-4b68-a358-3a4ed3f78c2a None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:48.279898 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5b1c1a-e452-4b68-a358-3a4ed3f78c2a None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:48.280177 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5b1c1a-e452-4b68-a358-3a4ed3f78c2a None admin] PATH_INFO: `/v3/users` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:48.281021 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5b1c1a-e452-4b68-a358-3a4ed3f78c2a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:47:48.281332 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5b1c1a-e452-4b68-a358-3a4ed3f78c2a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:48.282797 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5b1c1a-e452-4b68-a358-3a4ed3f78c2a None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:48.284196 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:48.310353 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5362ed83-cd6c-4001-bc86-2bd5d7e5296b None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:48.328971 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 237/492] 10.222.0.46 () {56 vars in 1133 bytes} [Thu Jun 23 15:47:48 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:47:48.328971 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5b1c1a-e452-4b68-a358-3a4ed3f78c2a None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:48.353164 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb27435e-d810-4d68-879b-1ab8759575d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:48.353885 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb27435e-d810-4d68-879b-1ab8759575d1 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:48.354168 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb27435e-d810-4d68-879b-1ab8759575d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:48.354440 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb27435e-d810-4d68-879b-1ab8759575d1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:48.365610 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb27435e-d810-4d68-879b-1ab8759575d1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 15:47:48.373701 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-fb27435e-d810-4d68-879b-1ab8759575d1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 23 15:47:48.373701 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:47:48.373701 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:47:48.373701 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:47:48.373701 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:47:48.373701 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:47:48.373701 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:47:48.373701 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:47:48.373701 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:47:48.373701 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:47:48.373701 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:47:48.373701 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:47:48.373701 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 15:47:48.373701 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 15:47:48.373701 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 15:47:48.373701 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 15:47:48.374643 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 15:47:48.374643 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 15:47:48.374643 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 15:47:48.374643 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 15:47:48.374643 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:47:48.374643 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:47:48.374643 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:47:48.374643 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:47:48.374643 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:47:48.374643 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:47:48.374643 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:47:48.374643 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Jun 23 15:47:48.374643 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:47:48.374643 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:47:48.374643 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:47:48.374643 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:47:48.375469 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:47:48.375469 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:47:48.375469 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:47:48.375469 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:47:48.375469 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 15:47:48.375469 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 15:47:48.375469 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 15:47:48.375469 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 15:47:48.375469 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 15:47:48.375469 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 15:47:48.375469 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 23 15:47:48.375469 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Jun 23 15:47:48.375469 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 256/493] 10.222.0.46 () {56 vars in 1102 bytes} [Thu Jun 23 15:47:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:47:48.376635 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eae69de-d4f9-461e-a745-332dcf028750 None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:48.376635 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eae69de-d4f9-461e-a745-332dcf028750 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:48.376878 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eae69de-d4f9-461e-a745-332dcf028750 None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:48.382894 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 257/494] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jun 23 15:47:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 15:47:48.382894 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eae69de-d4f9-461e-a745-332dcf028750 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:48.434064 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5b1c1a-e452-4b68-a358-3a4ed3f78c2a None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:48.468162 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99d4805-80e7-4fc7-8007-41661d67b031 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:48.468964 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99d4805-80e7-4fc7-8007-41661d67b031 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:48.469251 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99d4805-80e7-4fc7-8007-41661d67b031 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:48.469525 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99d4805-80e7-4fc7-8007-41661d67b031 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:48.480112 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99d4805-80e7-4fc7-8007-41661d67b031 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 15:47:48.488107 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-d99d4805-80e7-4fc7-8007-41661d67b031 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 23 15:47:48.488107 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:47:48.488107 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:47:48.488107 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:47:48.488107 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:47:48.488107 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:47:48.488107 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:47:48.488107 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:47:48.488107 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:47:48.488107 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:47:48.488107 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:47:48.488107 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:47:48.488107 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 15:47:48.488107 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 15:47:48.488107 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 15:47:48.488107 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 15:47:48.489098 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 15:47:48.489098 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 15:47:48.489098 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 15:47:48.489098 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 15:47:48.489098 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:47:48.489098 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:47:48.489098 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:47:48.489098 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:47:48.489098 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:47:48.489098 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:47:48.489098 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:47:48.489098 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Jun 23 15:47:48.489098 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:47:48.489098 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:47:48.489098 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:47:48.489098 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:47:48.489891 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:47:48.489891 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:47:48.489891 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:47:48.489891 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:47:48.489891 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 15:47:48.489891 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 15:47:48.489891 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 15:47:48.489891 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 15:47:48.489891 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 15:47:48.489891 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 15:47:48.489891 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 23 15:47:48.489891 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Jun 23 15:47:48.490495 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eae69de-d4f9-461e-a745-332dcf028750 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:48.490576 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 238/495] 10.222.0.46 () {56 vars in 1102 bytes} [Thu Jun 23 15:47:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:47:48.490576 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99d4805-80e7-4fc7-8007-41661d67b031 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:48.528325 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4decb3-e5a3-439b-8527-df5772d0645a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:48.528978 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4decb3-e5a3-439b-8527-df5772d0645a None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:48.529262 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4decb3-e5a3-439b-8527-df5772d0645a None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:48.529556 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4decb3-e5a3-439b-8527-df5772d0645a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:48.538249 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:48.543574 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:48.600604 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99d4805-80e7-4fc7-8007-41661d67b031 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:48.604987 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4decb3-e5a3-439b-8527-df5772d0645a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:48.606396 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4decb3-e5a3-439b-8527-df5772d0645a None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:48.634905 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ae018e-5846-4b1b-8a2c-d72cf9112028 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:48.635581 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ae018e-5846-4b1b-8a2c-d72cf9112028 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:48.635867 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ae018e-5846-4b1b-8a2c-d72cf9112028 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:48.636148 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ae018e-5846-4b1b-8a2c-d72cf9112028 None admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:48.636903 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ae018e-5846-4b1b-8a2c-d72cf9112028 None admin] RBAC: Adding query filter params (name=service) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:47:48.637212 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ae018e-5846-4b1b-8a2c-d72cf9112028 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:48.638622 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ae018e-5846-4b1b-8a2c-d72cf9112028 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:48.646415 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 239/496] 10.222.0.46 () {56 vars in 1103 bytes} [Thu Jun 23 15:47:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:47:48.647688 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed030db-8c29-4263-8b4f-3afe005597f8 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:48.647977 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed030db-8c29-4263-8b4f-3afe005597f8 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:48.648254 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed030db-8c29-4263-8b4f-3afe005597f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:48.664580 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-fed030db-8c29-4263-8b4f-3afe005597f8 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:47:48.665760 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-fed030db-8c29-4263-8b4f-3afe005597f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:47:48.710332 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 258/497] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Jun 23 15:47:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 15:47:48.711698 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5bba2b-46b4-467e-86fd-915ad8b2e09e None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:48.711981 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5bba2b-46b4-467e-86fd-915ad8b2e09e None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:48.712255 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5bba2b-46b4-467e-86fd-915ad8b2e09e None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:48.713513 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 259/498] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jun 23 15:47:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 15:47:48.714205 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5bba2b-46b4-467e-86fd-915ad8b2e09e None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:48.735942 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 240/499] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jun 23 15:47:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 15:47:48.736873 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed030db-8c29-4263-8b4f-3afe005597f8 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:48.769508 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:48.783737 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:48.839606 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5bba2b-46b4-467e-86fd-915ad8b2e09e None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:48.848794 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed030db-8c29-4263-8b4f-3afe005597f8 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:48.875455 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a596d1-e78b-49f6-90f5-2e57a20b990c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:48.876043 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a596d1-e78b-49f6-90f5-2e57a20b990c None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:48.876322 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a596d1-e78b-49f6-90f5-2e57a20b990c None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:48.876595 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a596d1-e78b-49f6-90f5-2e57a20b990c None admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:48.877336 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a596d1-e78b-49f6-90f5-2e57a20b990c None admin] RBAC: Adding query filter params (name=service) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:47:48.877644 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a596d1-e78b-49f6-90f5-2e57a20b990c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:48.879044 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a596d1-e78b-49f6-90f5-2e57a20b990c None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:48.883131 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746b412a-a87d-4ebb-8421-92e10abba512 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:48.883946 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746b412a-a87d-4ebb-8421-92e10abba512 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:48.884227 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746b412a-a87d-4ebb-8421-92e10abba512 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:48.884503 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746b412a-a87d-4ebb-8421-92e10abba512 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:48.887811 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 260/500] 10.222.0.46 () {56 vars in 1103 bytes} [Thu Jun 23 15:47:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:47:48.888624 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a596d1-e78b-49f6-90f5-2e57a20b990c None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:48.895563 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746b412a-a87d-4ebb-8421-92e10abba512 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:48.897013 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746b412a-a87d-4ebb-8421-92e10abba512 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:48.915183 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-746b412a-a87d-4ebb-8421-92e10abba512 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 15:47:48.915183 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:47:48.915183 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:47:48.915183 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:47:48.915183 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:47:48.915183 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:47:48.915183 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:47:48.915183 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:47:48.915183 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:47:48.915183 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:47:48.915183 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:47:48.915183 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:47:48.915183 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 15:47:48.915183 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 15:47:48.915183 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 15:47:48.916024 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 15:47:48.916024 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:47:48.916024 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:47:48.916024 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:47:48.916024 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:47:48.916024 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:47:48.916024 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:47:48.916024 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:47:48.916024 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Jun 23 15:47:48.916024 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:47:48.916024 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:47:48.916024 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:47:48.916024 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:47:48.916024 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:47:48.916024 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:47:48.916024 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:47:48.916024 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:47:48.916855 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 15:47:48.916855 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 15:47:48.916855 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 15:47:48.916855 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Jun 23 15:47:48.916855 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 241/501] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Jun 23 15:47:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:47:48.917688 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade18b0d-9cf2-4d19-91a5-00ce66d8ac04 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:48.917978 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade18b0d-9cf2-4d19-91a5-00ce66d8ac04 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:48.918256 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade18b0d-9cf2-4d19-91a5-00ce66d8ac04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:48.934636 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-ade18b0d-9cf2-4d19-91a5-00ce66d8ac04 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:47:48.935863 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-ade18b0d-9cf2-4d19-91a5-00ce66d8ac04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:47:48.986677 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:48.990524 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a596d1-e78b-49f6-90f5-2e57a20b990c None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:49.000714 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 242/502] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jun 23 15:47:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 15:47:49.001554 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade18b0d-9cf2-4d19-91a5-00ce66d8ac04 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:49.028435 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:49.029250 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c73ce8-24de-4839-b29c-66f0f341bd07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:49.030053 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c73ce8-24de-4839-b29c-66f0f341bd07 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:49.030340 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c73ce8-24de-4839-b29c-66f0f341bd07 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:49.030620 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c73ce8-24de-4839-b29c-66f0f341bd07 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:49.042743 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c73ce8-24de-4839-b29c-66f0f341bd07 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:49.044225 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c73ce8-24de-4839-b29c-66f0f341bd07 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:49.061272 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-00c73ce8-24de-4839-b29c-66f0f341bd07 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 15:47:49.061272 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:47:49.061272 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:47:49.061272 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:47:49.061272 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:47:49.061272 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:47:49.061272 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:47:49.061272 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:47:49.061272 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:47:49.061272 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:47:49.061272 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:47:49.061272 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:47:49.061272 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 15:47:49.061272 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 15:47:49.061272 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 15:47:49.062165 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 15:47:49.062165 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:47:49.062165 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:47:49.062165 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:47:49.062165 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:47:49.062165 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:47:49.062165 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:47:49.062165 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:47:49.062165 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Jun 23 15:47:49.062165 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:47:49.062165 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:47:49.062165 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:47:49.062165 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:47:49.062165 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:47:49.062165 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:47:49.062165 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:47:49.062165 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:47:49.063462 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 15:47:49.063462 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 15:47:49.063462 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 15:47:49.063462 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Jun 23 15:47:49.063462 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 261/503] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Jun 23 15:47:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:47:49.063462 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c73ce8-24de-4839-b29c-66f0f341bd07 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:49.114097 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade18b0d-9cf2-4d19-91a5-00ce66d8ac04 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:49.147510 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be48530a-a417-48aa-b38f-a9b68454e7ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:49.148301 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be48530a-a417-48aa-b38f-a9b68454e7ae None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:49.148520 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be48530a-a417-48aa-b38f-a9b68454e7ae None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:49.148790 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be48530a-a417-48aa-b38f-a9b68454e7ae None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:49.158929 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be48530a-a417-48aa-b38f-a9b68454e7ae None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:49.160393 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be48530a-a417-48aa-b38f-a9b68454e7ae None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:49.163940 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c73ce8-24de-4839-b29c-66f0f341bd07 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:49.177881 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-be48530a-a417-48aa-b38f-a9b68454e7ae None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 15:47:49.177881 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:47:49.177881 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:47:49.177881 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:47:49.177881 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:47:49.177881 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:47:49.177881 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:47:49.177881 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:47:49.177881 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:47:49.177881 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:47:49.177881 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:47:49.177881 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:47:49.177881 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 15:47:49.177881 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 15:47:49.177881 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 15:47:49.178767 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 15:47:49.178767 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:47:49.178767 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:47:49.178767 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:47:49.178767 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:47:49.178767 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:47:49.178767 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:47:49.178767 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:47:49.178767 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Jun 23 15:47:49.178767 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:47:49.178767 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:47:49.178767 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:47:49.178767 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:47:49.178767 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:47:49.178767 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:47:49.178767 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:47:49.178767 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:47:49.179721 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 15:47:49.179721 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 15:47:49.179721 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 15:47:49.179721 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Jun 23 15:47:49.182850 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 243/504] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Jun 23 15:47:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:47:49.183925 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be48530a-a417-48aa-b38f-a9b68454e7ae None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:49.198261 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705484fa-830c-426c-92bf-25b509de9d69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:49.199091 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705484fa-830c-426c-92bf-25b509de9d69 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:49.199373 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705484fa-830c-426c-92bf-25b509de9d69 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:49.199649 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705484fa-830c-426c-92bf-25b509de9d69 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:49.210912 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705484fa-830c-426c-92bf-25b509de9d69 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:49.212394 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705484fa-830c-426c-92bf-25b509de9d69 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:49.231274 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-705484fa-830c-426c-92bf-25b509de9d69 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 15:47:49.231274 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:47:49.231274 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:47:49.231274 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:47:49.231274 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:47:49.231274 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:47:49.231274 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:47:49.231274 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:47:49.231274 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:47:49.231274 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:47:49.231274 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:47:49.231274 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:47:49.231274 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 15:47:49.231274 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 15:47:49.231274 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 15:47:49.232407 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 15:47:49.232407 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:47:49.232407 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:47:49.232407 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:47:49.232407 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:47:49.232407 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:47:49.232407 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:47:49.232407 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:47:49.232407 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Jun 23 15:47:49.232407 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:47:49.232407 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:47:49.232407 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:47:49.232407 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:47:49.232407 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:47:49.232407 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:47:49.232407 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:47:49.232407 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:47:49.233242 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 15:47:49.233242 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 15:47:49.233242 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 15:47:49.233242 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Jun 23 15:47:49.233242 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 262/505] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Jun 23 15:47:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:47:49.233242 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705484fa-830c-426c-92bf-25b509de9d69 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:49.236367 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:49.261679 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:49.293607 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be48530a-a417-48aa-b38f-a9b68454e7ae None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:49.327062 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d720e1e0-3940-44b2-90af-a3228c2c3bf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:49.327709 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d720e1e0-3940-44b2-90af-a3228c2c3bf4 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:49.327997 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d720e1e0-3940-44b2-90af-a3228c2c3bf4 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:49.328274 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d720e1e0-3940-44b2-90af-a3228c2c3bf4 None admin] PATH_INFO: `/v3/domains` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:49.329007 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d720e1e0-3940-44b2-90af-a3228c2c3bf4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:47:49.329317 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d720e1e0-3940-44b2-90af-a3228c2c3bf4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:49.330728 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d720e1e0-3940-44b2-90af-a3228c2c3bf4 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:49.337192 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705484fa-830c-426c-92bf-25b509de9d69 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:49.342461 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 244/506] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Jun 23 15:47:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:47:49.343135 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d720e1e0-3940-44b2-90af-a3228c2c3bf4 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:49.374288 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d34a0ca-7b16-48c7-abcb-d4c150eb9370 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:49.375123 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d34a0ca-7b16-48c7-abcb-d4c150eb9370 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:49.375411 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d34a0ca-7b16-48c7-abcb-d4c150eb9370 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:49.375689 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d34a0ca-7b16-48c7-abcb-d4c150eb9370 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:49.387695 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d34a0ca-7b16-48c7-abcb-d4c150eb9370 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 15:47:49.395418 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-0d34a0ca-7b16-48c7-abcb-d4c150eb9370 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 23 15:47:49.395418 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:47:49.395418 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:47:49.395418 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:47:49.395418 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:47:49.395418 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:47:49.395418 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:47:49.395418 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:47:49.395418 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:47:49.395418 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:47:49.395418 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:47:49.395418 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:47:49.395418 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 15:47:49.395418 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 15:47:49.395418 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 15:47:49.395418 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 15:47:49.396312 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 15:47:49.396312 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 15:47:49.396312 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 15:47:49.396312 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 15:47:49.396312 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:47:49.396312 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:47:49.396312 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:47:49.396312 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:47:49.396312 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:47:49.396312 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:47:49.396312 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:47:49.396312 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Jun 23 15:47:49.396312 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:47:49.396312 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:47:49.396312 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:47:49.396312 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:47:49.397878 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:47:49.397878 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:47:49.397878 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:47:49.397878 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:47:49.397878 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 15:47:49.397878 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 15:47:49.397878 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 15:47:49.397878 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 15:47:49.397878 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 15:47:49.397878 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 15:47:49.397878 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 23 15:47:49.397878 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Jun 23 15:47:49.397878 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 263/507] 10.222.0.46 () {56 vars in 1102 bytes} [Thu Jun 23 15:47:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:47:49.399009 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d34a0ca-7b16-48c7-abcb-d4c150eb9370 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:49.449142 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d720e1e0-3940-44b2-90af-a3228c2c3bf4 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:49.459495 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:49.477754 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:49.483643 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac82ce13-6457-4e4b-934d-2424080a517f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:49.484317 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac82ce13-6457-4e4b-934d-2424080a517f None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:49.484605 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac82ce13-6457-4e4b-934d-2424080a517f None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:49.484883 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac82ce13-6457-4e4b-934d-2424080a517f None admin] PATH_INFO: `/v3/domains` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:49.485622 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac82ce13-6457-4e4b-934d-2424080a517f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:47:49.485925 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac82ce13-6457-4e4b-934d-2424080a517f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:49.487328 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac82ce13-6457-4e4b-934d-2424080a517f None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:49.498786 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 245/508] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Jun 23 15:47:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:47:49.499182 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d34a0ca-7b16-48c7-abcb-d4c150eb9370 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:49.499582 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac82ce13-6457-4e4b-934d-2424080a517f None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:49.534802 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff201630-8788-44cb-a75f-dbac479efabd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:49.535461 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff201630-8788-44cb-a75f-dbac479efabd None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:49.535750 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff201630-8788-44cb-a75f-dbac479efabd None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:49.536024 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff201630-8788-44cb-a75f-dbac479efabd None admin] PATH_INFO: `/v3/domains` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:49.536753 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff201630-8788-44cb-a75f-dbac479efabd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:47:49.537062 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff201630-8788-44cb-a75f-dbac479efabd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:49.538476 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff201630-8788-44cb-a75f-dbac479efabd None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:49.550330 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 264/509] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Jun 23 15:47:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:47:49.551168 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff201630-8788-44cb-a75f-dbac479efabd None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:49.608089 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac82ce13-6457-4e4b-934d-2424080a517f None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:49.641553 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3b8610-216e-406a-877c-daac4efa60c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:49.642239 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3b8610-216e-406a-877c-daac4efa60c8 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:49.642508 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3b8610-216e-406a-877c-daac4efa60c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:49.642821 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3b8610-216e-406a-877c-daac4efa60c8 None admin] PATH_INFO: `/v3/domains` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:49.643544 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3b8610-216e-406a-877c-daac4efa60c8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:47:49.643851 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3b8610-216e-406a-877c-daac4efa60c8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:49.645213 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3b8610-216e-406a-877c-daac4efa60c8 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:49.649662 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff201630-8788-44cb-a75f-dbac479efabd None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:49.657315 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 246/510] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Jun 23 15:47:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:47:49.658127 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3b8610-216e-406a-877c-daac4efa60c8 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:49.672401 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:49.685410 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba688af4-0d96-4805-8326-ea86ffe0b1b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:49.686465 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba688af4-0d96-4805-8326-ea86ffe0b1b3 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:49.686784 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba688af4-0d96-4805-8326-ea86ffe0b1b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:49.687076 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba688af4-0d96-4805-8326-ea86ffe0b1b3 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:49.698053 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba688af4-0d96-4805-8326-ea86ffe0b1b3 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:49.699543 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba688af4-0d96-4805-8326-ea86ffe0b1b3 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:49.712896 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:49.716392 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-ba688af4-0d96-4805-8326-ea86ffe0b1b3 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 23 15:47:49.716392 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:47:49.716392 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:47:49.716392 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:47:49.716392 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:47:49.716392 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:47:49.716392 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:47:49.716392 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:47:49.716392 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:47:49.716392 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:47:49.716392 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:47:49.716392 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:47:49.716392 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 15:47:49.716392 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 15:47:49.716392 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 15:47:49.716392 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 15:47:49.717325 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:47:49.717325 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:47:49.717325 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 15:47:49.717325 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 15:47:49.717325 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 15:47:49.717325 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 15:47:49.717325 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:47:49.717325 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:47:49.717325 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:47:49.717325 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:47:49.717325 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:47:49.717325 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Jun 23 15:47:49.717325 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:47:49.717325 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:47:49.717325 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:47:49.717325 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:47:49.717325 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:47:49.717325 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:47:49.718233 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:47:49.718233 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:47:49.718233 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 15:47:49.718233 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 15:47:49.718233 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 15:47:49.718233 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 15:47:49.718233 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 15:47:49.718233 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 15:47:49.718233 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 23 15:47:49.718233 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Jun 23 15:47:49.718233 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 265/511] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Jun 23 15:47:49 2022] GET /identity/v3/users/cinder => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:47:49.718233 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba688af4-0d96-4805-8326-ea86ffe0b1b3 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:49.766852 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3b8610-216e-406a-877c-daac4efa60c8 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:49.799747 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978a2dce-3d00-47d4-81cc-8722c9c691c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:49.800426 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978a2dce-3d00-47d4-81cc-8722c9c691c5 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:49.800713 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978a2dce-3d00-47d4-81cc-8722c9c691c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:49.800996 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978a2dce-3d00-47d4-81cc-8722c9c691c5 None admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:49.801748 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978a2dce-3d00-47d4-81cc-8722c9c691c5 None admin] RBAC: Adding query filter params (name=service) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:47:49.802060 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978a2dce-3d00-47d4-81cc-8722c9c691c5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:49.803983 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978a2dce-3d00-47d4-81cc-8722c9c691c5 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:49.815144 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 247/512] 10.222.0.46 () {56 vars in 1103 bytes} [Thu Jun 23 15:47:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:47:49.815144 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978a2dce-3d00-47d4-81cc-8722c9c691c5 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:49.821506 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba688af4-0d96-4805-8326-ea86ffe0b1b3 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:49.857075 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1127ae4-d9dc-4dd5-b571-1a10d3c10b12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:49.857871 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1127ae4-d9dc-4dd5-b571-1a10d3c10b12 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:49.858157 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1127ae4-d9dc-4dd5-b571-1a10d3c10b12 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:49.858435 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1127ae4-d9dc-4dd5-b571-1a10d3c10b12 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:49.865353 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1127ae4-d9dc-4dd5-b571-1a10d3c10b12 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:49.867492 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1127ae4-d9dc-4dd5-b571-1a10d3c10b12 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:49.882937 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-c1127ae4-d9dc-4dd5-b571-1a10d3c10b12 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 15:47:49.882937 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:47:49.882937 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:47:49.882937 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:47:49.882937 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:47:49.882937 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:47:49.882937 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:47:49.882937 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:47:49.882937 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:47:49.882937 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:47:49.882937 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:47:49.882937 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:47:49.882937 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 15:47:49.882937 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 15:47:49.882937 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 15:47:49.883772 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 15:47:49.883772 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:47:49.883772 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:47:49.883772 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:47:49.883772 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:47:49.883772 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:47:49.883772 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:47:49.883772 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:47:49.883772 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Jun 23 15:47:49.883772 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:47:49.883772 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:47:49.883772 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:47:49.883772 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:47:49.883772 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:47:49.883772 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:47:49.883772 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:47:49.883772 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:47:49.884594 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 15:47:49.884594 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 15:47:49.884594 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 15:47:49.884594 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 15:47:49.884594 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 15:47:49.884594 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 15:47:49.884594 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 15:47:49.884594 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Jun 23 15:47:49.884594 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 266/513] 10.222.0.46 () {56 vars in 1111 bytes} [Thu Jun 23 15:47:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:47:49.885074 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1127ae4-d9dc-4dd5-b571-1a10d3c10b12 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:49.913873 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978a2dce-3d00-47d4-81cc-8722c9c691c5 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:49.926345 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:49.927835 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:49.948648 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57f0699-7a00-4c62-bc1d-ab47fbac1944 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:49.949332 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57f0699-7a00-4c62-bc1d-ab47fbac1944 None admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:49.949623 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57f0699-7a00-4c62-bc1d-ab47fbac1944 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:49.949902 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57f0699-7a00-4c62-bc1d-ab47fbac1944 None admin] PATH_INFO: `/v3/users` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:49.950608 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57f0699-7a00-4c62-bc1d-ab47fbac1944 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:49.952089 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57f0699-7a00-4c62-bc1d-ab47fbac1944 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:49.972237 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 248/514] 10.222.0.46 () {60 vars in 1132 bytes} [Thu Jun 23 15:47:49 2022] POST /identity/v3/users => generated 247 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 15:47:49.973072 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57f0699-7a00-4c62-bc1d-ab47fbac1944 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:49.991772 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1127ae4-d9dc-4dd5-b571-1a10d3c10b12 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:50.034085 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c687b5-4c79-4ef7-8608-5265b85a733a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:50.034928 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c687b5-4c79-4ef7-8608-5265b85a733a None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:50.035219 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c687b5-4c79-4ef7-8608-5265b85a733a None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:50.035498 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c687b5-4c79-4ef7-8608-5265b85a733a None admin] PATH_INFO: `/v3/users/glance` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:50.045122 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c687b5-4c79-4ef7-8608-5265b85a733a None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:50.046736 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c687b5-4c79-4ef7-8608-5265b85a733a None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:50.062584 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-25c687b5-4c79-4ef7-8608-5265b85a733a None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 23 15:47:50.062584 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:47:50.062584 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:47:50.062584 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:47:50.062584 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:47:50.062584 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:47:50.062584 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:47:50.062584 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:47:50.062584 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:47:50.062584 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:47:50.062584 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:47:50.062584 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:47:50.062584 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 15:47:50.062584 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 15:47:50.062584 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 15:47:50.062584 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 15:47:50.064085 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:47:50.064085 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:47:50.064085 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 15:47:50.064085 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 15:47:50.064085 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 15:47:50.064085 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 15:47:50.064085 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:47:50.064085 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:47:50.064085 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:47:50.064085 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:47:50.064085 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:47:50.064085 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Jun 23 15:47:50.064085 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:47:50.064085 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:47:50.064085 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:47:50.064085 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:47:50.064085 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:47:50.064085 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:47:50.064970 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:47:50.064970 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:47:50.064970 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 15:47:50.064970 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 15:47:50.064970 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 15:47:50.064970 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 15:47:50.064970 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 15:47:50.064970 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 15:47:50.064970 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 23 15:47:50.064970 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Jun 23 15:47:50.064970 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 267/515] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Jun 23 15:47:49 2022] GET /identity/v3/users/glance => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:47:50.064970 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c687b5-4c79-4ef7-8608-5265b85a733a None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:50.111179 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57f0699-7a00-4c62-bc1d-ab47fbac1944 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:50.151314 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2311dcd0-87e6-48a8-bbb7-723e25a2a2c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:50.152103 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2311dcd0-87e6-48a8-bbb7-723e25a2a2c9 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:50.152390 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2311dcd0-87e6-48a8-bbb7-723e25a2a2c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:50.153233 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2311dcd0-87e6-48a8-bbb7-723e25a2a2c9 None admin] PATH_INFO: `/v3/users` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:50.153493 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2311dcd0-87e6-48a8-bbb7-723e25a2a2c9 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:47:50.154115 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2311dcd0-87e6-48a8-bbb7-723e25a2a2c9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:50.155298 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2311dcd0-87e6-48a8-bbb7-723e25a2a2c9 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:50.177141 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 249/516] 10.222.0.46 () {56 vars in 1137 bytes} [Thu Jun 23 15:47:49 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:47:50.178863 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2311dcd0-87e6-48a8-bbb7-723e25a2a2c9 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:50.187319 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c687b5-4c79-4ef7-8608-5265b85a733a None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:50.203558 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:50.209778 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:50.223092 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3f0fe1-ac41-43a2-9389-e158375d5901 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:50.223890 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3f0fe1-ac41-43a2-9389-e158375d5901 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:50.224169 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3f0fe1-ac41-43a2-9389-e158375d5901 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:50.224440 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3f0fe1-ac41-43a2-9389-e158375d5901 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:50.237084 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3f0fe1-ac41-43a2-9389-e158375d5901 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:50.238538 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3f0fe1-ac41-43a2-9389-e158375d5901 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:50.257203 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-4f3f0fe1-ac41-43a2-9389-e158375d5901 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 15:47:50.257203 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:47:50.257203 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:47:50.257203 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:47:50.257203 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:47:50.257203 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:47:50.257203 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:47:50.257203 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:47:50.257203 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:47:50.257203 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:47:50.257203 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:47:50.257203 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:47:50.257203 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 15:47:50.257203 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 15:47:50.257203 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 15:47:50.258256 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 15:47:50.258256 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:47:50.258256 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:47:50.258256 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:47:50.258256 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:47:50.258256 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:47:50.258256 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:47:50.258256 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:47:50.258256 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Jun 23 15:47:50.258256 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:47:50.258256 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:47:50.258256 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:47:50.258256 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:47:50.258256 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:47:50.258256 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:47:50.258256 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:47:50.258256 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:47:50.259322 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 15:47:50.259322 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 15:47:50.259322 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 15:47:50.259322 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Jun 23 15:47:50.259322 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 268/517] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Jun 23 15:47:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:47:50.259628 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3f0fe1-ac41-43a2-9389-e158375d5901 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:50.294622 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2311dcd0-87e6-48a8-bbb7-723e25a2a2c9 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:50.331061 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056a7024-27cb-4c30-b4c8-475d0482ed67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:50.331735 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056a7024-27cb-4c30-b4c8-475d0482ed67 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:50.332016 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056a7024-27cb-4c30-b4c8-475d0482ed67 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:50.332292 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056a7024-27cb-4c30-b4c8-475d0482ed67 None admin] PATH_INFO: `/v3/projects` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:50.333080 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056a7024-27cb-4c30-b4c8-475d0482ed67 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:47:50.333392 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056a7024-27cb-4c30-b4c8-475d0482ed67 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:50.334895 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056a7024-27cb-4c30-b4c8-475d0482ed67 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:50.353114 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 250/518] 10.222.0.46 () {56 vars in 1148 bytes} [Thu Jun 23 15:47:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:47:50.353952 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056a7024-27cb-4c30-b4c8-475d0482ed67 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:50.370334 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3f0fe1-ac41-43a2-9389-e158375d5901 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:50.405908 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f32f3f5-4816-4283-b507-4d0bc35b35e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:50.406718 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f32f3f5-4816-4283-b507-4d0bc35b35e0 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:50.407017 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f32f3f5-4816-4283-b507-4d0bc35b35e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:50.407299 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f32f3f5-4816-4283-b507-4d0bc35b35e0 None admin] PATH_INFO: `/v3/users` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:50.408143 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f32f3f5-4816-4283-b507-4d0bc35b35e0 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:47:50.408457 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f32f3f5-4816-4283-b507-4d0bc35b35e0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:50.410028 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f32f3f5-4816-4283-b507-4d0bc35b35e0 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:50.450670 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:50.452833 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 269/519] 10.222.0.46 () {56 vars in 1137 bytes} [Thu Jun 23 15:47:50 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:47:50.453768 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f32f3f5-4816-4283-b507-4d0bc35b35e0 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:50.466954 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056a7024-27cb-4c30-b4c8-475d0482ed67 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:50.471058 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:50.504263 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf607a1-68a6-4670-9f88-e6ad7e0f2fad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:50.504930 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf607a1-68a6-4670-9f88-e6ad7e0f2fad None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:50.505220 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf607a1-68a6-4670-9f88-e6ad7e0f2fad None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:50.505503 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf607a1-68a6-4670-9f88-e6ad7e0f2fad None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:50.575298 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f32f3f5-4816-4283-b507-4d0bc35b35e0 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:50.578946 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf607a1-68a6-4670-9f88-e6ad7e0f2fad None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:50.580436 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf607a1-68a6-4670-9f88-e6ad7e0f2fad None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:50.612731 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5d02c9-f317-4ea4-a28d-f459c3fe8aa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:50.613405 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5d02c9-f317-4ea4-a28d-f459c3fe8aa4 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:50.613702 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5d02c9-f317-4ea4-a28d-f459c3fe8aa4 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:50.613962 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5d02c9-f317-4ea4-a28d-f459c3fe8aa4 None admin] PATH_INFO: `/v3/domains` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:50.614708 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5d02c9-f317-4ea4-a28d-f459c3fe8aa4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:47:50.615022 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5d02c9-f317-4ea4-a28d-f459c3fe8aa4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:50.616619 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5d02c9-f317-4ea4-a28d-f459c3fe8aa4 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:50.628741 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 270/520] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Jun 23 15:47:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:47:50.629593 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5d02c9-f317-4ea4-a28d-f459c3fe8aa4 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:50.676326 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:50.694280 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 251/521] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Jun 23 15:47:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 15:47:50.699457 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf607a1-68a6-4670-9f88-e6ad7e0f2fad None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:50.703456 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:50.742545 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5d02c9-f317-4ea4-a28d-f459c3fe8aa4 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:50.783818 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d49338e-2828-4e3e-a8fd-5f7bfa882af4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:50.784512 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d49338e-2828-4e3e-a8fd-5f7bfa882af4 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:50.784802 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d49338e-2828-4e3e-a8fd-5f7bfa882af4 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:50.785081 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d49338e-2828-4e3e-a8fd-5f7bfa882af4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:50.785953 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d49338e-2828-4e3e-a8fd-5f7bfa882af4 None admin] RBAC: Adding query filter params (role.id=bfa8ecadc3d5483e821420aa0f0fb41e, scope.project.id=28ecdf5b474c44098c722f677d2a0b77, user.id=705115d19bab424d894bea8b1b48a7f0) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:47:50.787361 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d49338e-2828-4e3e-a8fd-5f7bfa882af4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:50.788547 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d49338e-2828-4e3e-a8fd-5f7bfa882af4 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:50.803685 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 271/522] 10.222.0.46 () {56 vars in 1374 bytes} [Thu Jun 23 15:47:50 2022] GET /identity/v3/role_assignments?user.id=705115d19bab424d894bea8b1b48a7f0&scope.project.id=28ecdf5b474c44098c722f677d2a0b77&role.id=bfa8ecadc3d5483e821420aa0f0fb41e => generated 259 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:47:50.803685 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d49338e-2828-4e3e-a8fd-5f7bfa882af4 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:50.839276 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf607a1-68a6-4670-9f88-e6ad7e0f2fad None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:50.888234 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc296e3-186f-40c4-80ae-45cbe8cd087d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:50.889186 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc296e3-186f-40c4-80ae-45cbe8cd087d None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:50.890418 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc296e3-186f-40c4-80ae-45cbe8cd087d None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:50.890740 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc296e3-186f-40c4-80ae-45cbe8cd087d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:50.931042 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d49338e-2828-4e3e-a8fd-5f7bfa882af4 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:50.973173 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:50.978310 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f69f5b-fabc-4706-828b-aaad0877d3bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:50.979157 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f69f5b-fabc-4706-828b-aaad0877d3bb None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:50.979443 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f69f5b-fabc-4706-828b-aaad0877d3bb None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:50.979716 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f69f5b-fabc-4706-828b-aaad0877d3bb None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:50.983939 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:50.989738 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc296e3-186f-40c4-80ae-45cbe8cd087d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:50.989889 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f69f5b-fabc-4706-828b-aaad0877d3bb None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:50.991590 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f69f5b-fabc-4706-828b-aaad0877d3bb None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:50.992218 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc296e3-186f-40c4-80ae-45cbe8cd087d None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:51.009900 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-c2f69f5b-fabc-4706-828b-aaad0877d3bb None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 23 15:47:51.009900 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:47:51.009900 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:47:51.009900 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:47:51.009900 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:47:51.009900 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:47:51.009900 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:47:51.009900 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:47:51.009900 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:47:51.009900 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:47:51.009900 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:47:51.009900 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:47:51.009900 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 15:47:51.009900 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 15:47:51.009900 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 15:47:51.009900 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 15:47:51.011567 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:47:51.011567 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:47:51.011567 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 15:47:51.011567 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 15:47:51.011567 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 15:47:51.011567 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 15:47:51.011567 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:47:51.011567 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:47:51.011567 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:47:51.011567 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:47:51.011567 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:47:51.011567 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Jun 23 15:47:51.011567 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:47:51.011567 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:47:51.011567 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:47:51.011567 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:47:51.011567 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:47:51.011567 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:47:51.012616 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:47:51.012616 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:47:51.012616 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 15:47:51.012616 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 15:47:51.012616 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 15:47:51.012616 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 15:47:51.012616 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 15:47:51.012616 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 15:47:51.012616 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 23 15:47:51.012616 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Jun 23 15:47:51.012616 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 272/523] 10.222.0.46 () {56 vars in 1102 bytes} [Thu Jun 23 15:47:50 2022] GET /identity/v3/users/neutron => generated 85 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:47:51.012616 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f69f5b-fabc-4706-828b-aaad0877d3bb None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:51.112845 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 252/524] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Jun 23 15:47:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 15:47:51.114128 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc296e3-186f-40c4-80ae-45cbe8cd087d None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:51.129551 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f69f5b-fabc-4706-828b-aaad0877d3bb None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:51.168803 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcbd81a-a829-47ef-9a1b-d4f0c2d81c19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:51.169745 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcbd81a-a829-47ef-9a1b-d4f0c2d81c19 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:51.170096 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcbd81a-a829-47ef-9a1b-d4f0c2d81c19 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:51.170443 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcbd81a-a829-47ef-9a1b-d4f0c2d81c19 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:51.182600 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bcbd81a-a829-47ef-9a1b-d4f0c2d81c19 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:51.184321 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bcbd81a-a829-47ef-9a1b-d4f0c2d81c19 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:51.203295 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-2bcbd81a-a829-47ef-9a1b-d4f0c2d81c19 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 15:47:51.203295 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:47:51.203295 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:47:51.203295 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:47:51.203295 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:47:51.203295 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:47:51.203295 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:47:51.203295 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:47:51.203295 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:47:51.203295 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:47:51.203295 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:47:51.203295 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:47:51.203295 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 15:47:51.203295 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 15:47:51.203295 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 15:47:51.204391 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 15:47:51.204391 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:47:51.204391 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:47:51.204391 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:47:51.204391 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:47:51.204391 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:47:51.204391 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:47:51.204391 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:47:51.204391 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Jun 23 15:47:51.204391 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:47:51.204391 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:47:51.204391 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:47:51.204391 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:47:51.204391 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:47:51.204391 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:47:51.204391 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:47:51.204391 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:47:51.205433 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 15:47:51.205433 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 15:47:51.205433 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 15:47:51.205433 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Jun 23 15:47:51.206516 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 273/525] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Jun 23 15:47:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:47:51.207514 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcbd81a-a829-47ef-9a1b-d4f0c2d81c19 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:51.214385 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:51.232017 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc296e3-186f-40c4-80ae-45cbe8cd087d None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:51.264907 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:51.271652 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1eb9ce9-85f0-4a32-b43b-7a9beb7eca76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:51.273296 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1eb9ce9-85f0-4a32-b43b-7a9beb7eca76 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:51.273804 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1eb9ce9-85f0-4a32-b43b-7a9beb7eca76 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:51.274308 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1eb9ce9-85f0-4a32-b43b-7a9beb7eca76 None admin] PATH_INFO: `/v3/users` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:51.275750 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1eb9ce9-85f0-4a32-b43b-7a9beb7eca76 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:47:51.276268 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1eb9ce9-85f0-4a32-b43b-7a9beb7eca76 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:51.278131 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1eb9ce9-85f0-4a32-b43b-7a9beb7eca76 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:51.297681 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 253/526] 10.222.0.46 () {56 vars in 1139 bytes} [Thu Jun 23 15:47:51 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:47:51.298942 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1eb9ce9-85f0-4a32-b43b-7a9beb7eca76 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:51.337008 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcbd81a-a829-47ef-9a1b-d4f0c2d81c19 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:51.382383 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49a31f5-500e-403e-b56e-4700d24a4cc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:51.383205 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49a31f5-500e-403e-b56e-4700d24a4cc8 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:51.383491 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49a31f5-500e-403e-b56e-4700d24a4cc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:51.383765 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49a31f5-500e-403e-b56e-4700d24a4cc8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:51.396185 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49a31f5-500e-403e-b56e-4700d24a4cc8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:51.398164 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49a31f5-500e-403e-b56e-4700d24a4cc8 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:51.417704 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-f49a31f5-500e-403e-b56e-4700d24a4cc8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 15:47:51.417704 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:47:51.417704 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:47:51.417704 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:47:51.417704 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:47:51.417704 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:47:51.417704 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:47:51.417704 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:47:51.417704 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:47:51.417704 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:47:51.417704 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:47:51.417704 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:47:51.417704 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 15:47:51.417704 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 15:47:51.417704 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 15:47:51.418600 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 15:47:51.418600 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:47:51.418600 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:47:51.418600 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:47:51.418600 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:47:51.418600 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:47:51.418600 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:47:51.418600 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:47:51.418600 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Jun 23 15:47:51.418600 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:47:51.418600 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:47:51.418600 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:47:51.418600 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:47:51.418600 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:47:51.418600 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:47:51.418600 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:47:51.418600 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:47:51.420062 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 15:47:51.420062 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 15:47:51.420062 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 15:47:51.420062 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Jun 23 15:47:51.420062 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 274/527] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Jun 23 15:47:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:47:51.420427 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49a31f5-500e-403e-b56e-4700d24a4cc8 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:51.433614 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1eb9ce9-85f0-4a32-b43b-7a9beb7eca76 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:51.481374 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08f2961-ac63-49b2-be17-6d91c0492b89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:51.482666 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08f2961-ac63-49b2-be17-6d91c0492b89 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:51.483139 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08f2961-ac63-49b2-be17-6d91c0492b89 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:51.483562 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08f2961-ac63-49b2-be17-6d91c0492b89 None admin] PATH_INFO: `/v3/domains` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:51.484454 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08f2961-ac63-49b2-be17-6d91c0492b89 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:47:51.485421 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08f2961-ac63-49b2-be17-6d91c0492b89 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:51.487308 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08f2961-ac63-49b2-be17-6d91c0492b89 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:51.504874 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 254/528] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Jun 23 15:47:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:47:51.506011 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08f2961-ac63-49b2-be17-6d91c0492b89 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:51.509911 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:51.540920 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:51.544729 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49a31f5-500e-403e-b56e-4700d24a4cc8 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:51.587462 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9960f118-ff21-42b6-bad6-c99309287381 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:51.588275 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9960f118-ff21-42b6-bad6-c99309287381 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:51.588659 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9960f118-ff21-42b6-bad6-c99309287381 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:51.589034 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9960f118-ff21-42b6-bad6-c99309287381 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:51.648006 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08f2961-ac63-49b2-be17-6d91c0492b89 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:51.661915 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9960f118-ff21-42b6-bad6-c99309287381 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:51.663931 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9960f118-ff21-42b6-bad6-c99309287381 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:51.692888 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6414ab1-55d0-4bc5-8d99-16a4b2e905c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:51.694333 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6414ab1-55d0-4bc5-8d99-16a4b2e905c1 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:51.694910 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6414ab1-55d0-4bc5-8d99-16a4b2e905c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:51.695353 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6414ab1-55d0-4bc5-8d99-16a4b2e905c1 None admin] PATH_INFO: `/v3/domains` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:51.696326 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6414ab1-55d0-4bc5-8d99-16a4b2e905c1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:47:51.696774 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6414ab1-55d0-4bc5-8d99-16a4b2e905c1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:51.698270 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6414ab1-55d0-4bc5-8d99-16a4b2e905c1 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:51.712052 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 255/529] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Jun 23 15:47:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:47:51.713396 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6414ab1-55d0-4bc5-8d99-16a4b2e905c1 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:51.766744 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:51.776679 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 275/530] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Jun 23 15:47:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 15:47:51.777758 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9960f118-ff21-42b6-bad6-c99309287381 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:51.784067 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:51.826772 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6414ab1-55d0-4bc5-8d99-16a4b2e905c1 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:51.865408 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840ea2ed-c40f-4c97-956c-9599a099c15e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:51.866666 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840ea2ed-c40f-4c97-956c-9599a099c15e None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:51.867144 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840ea2ed-c40f-4c97-956c-9599a099c15e None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:51.867595 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840ea2ed-c40f-4c97-956c-9599a099c15e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:51.875278 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840ea2ed-c40f-4c97-956c-9599a099c15e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:51.877210 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840ea2ed-c40f-4c97-956c-9599a099c15e None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:51.882491 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9960f118-ff21-42b6-bad6-c99309287381 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:51.893527 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-840ea2ed-c40f-4c97-956c-9599a099c15e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 15:47:51.893527 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:47:51.893527 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:47:51.893527 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:47:51.893527 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:47:51.893527 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:47:51.893527 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:47:51.893527 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:47:51.893527 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:47:51.893527 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:47:51.893527 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:47:51.893527 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:47:51.893527 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 15:47:51.893527 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 15:47:51.893527 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 15:47:51.894398 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 15:47:51.894398 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:47:51.894398 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:47:51.894398 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:47:51.894398 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:47:51.894398 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:47:51.894398 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:47:51.894398 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:47:51.894398 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Jun 23 15:47:51.894398 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:47:51.894398 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:47:51.894398 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:47:51.894398 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:47:51.894398 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:47:51.894398 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:47:51.894398 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:47:51.894398 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:47:51.895278 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 15:47:51.895278 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 15:47:51.895278 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 15:47:51.895278 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 15:47:51.895278 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 15:47:51.895278 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 15:47:51.895278 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 15:47:51.895278 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Jun 23 15:47:51.898551 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 256/531] 10.222.0.46 () {56 vars in 1111 bytes} [Thu Jun 23 15:47:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:47:51.899620 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840ea2ed-c40f-4c97-956c-9599a099c15e None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:51.924582 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8443e0-650c-4df5-aad1-1181ef629a75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:51.925557 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8443e0-650c-4df5-aad1-1181ef629a75 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:51.925826 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8443e0-650c-4df5-aad1-1181ef629a75 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:51.926170 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8443e0-650c-4df5-aad1-1181ef629a75 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:51.933623 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8443e0-650c-4df5-aad1-1181ef629a75 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:51.935313 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8443e0-650c-4df5-aad1-1181ef629a75 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:51.950596 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-fe8443e0-650c-4df5-aad1-1181ef629a75 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 15:47:51.950596 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:47:51.950596 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:47:51.950596 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:47:51.950596 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:47:51.950596 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:47:51.950596 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:47:51.950596 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:47:51.950596 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:47:51.950596 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:47:51.950596 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:47:51.950596 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:47:51.950596 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 15:47:51.950596 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 15:47:51.950596 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 15:47:51.951544 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 15:47:51.951544 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:47:51.951544 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:47:51.951544 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:47:51.951544 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:47:51.951544 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:47:51.951544 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:47:51.951544 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:47:51.951544 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Jun 23 15:47:51.951544 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:47:51.951544 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:47:51.951544 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:47:51.951544 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:47:51.951544 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:47:51.951544 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:47:51.951544 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:47:51.951544 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:47:51.952603 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 15:47:51.952603 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 15:47:51.952603 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 15:47:51.952603 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 15:47:51.952603 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 15:47:51.952603 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 15:47:51.952603 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 15:47:51.952603 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Jun 23 15:47:51.952603 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 276/532] 10.222.0.46 () {56 vars in 1111 bytes} [Thu Jun 23 15:47:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:47:51.953736 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8443e0-650c-4df5-aad1-1181ef629a75 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:52.017642 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:52.020133 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840ea2ed-c40f-4c97-956c-9599a099c15e None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:52.046678 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:52.059738 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc22c7f-efab-48e3-aacc-c95ea8eafa16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:52.060913 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc22c7f-efab-48e3-aacc-c95ea8eafa16 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:52.061409 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc22c7f-efab-48e3-aacc-c95ea8eafa16 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:52.061855 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc22c7f-efab-48e3-aacc-c95ea8eafa16 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:52.063758 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8443e0-650c-4df5-aad1-1181ef629a75 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:52.074478 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc22c7f-efab-48e3-aacc-c95ea8eafa16 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:52.076313 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc22c7f-efab-48e3-aacc-c95ea8eafa16 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:52.095222 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-9fc22c7f-efab-48e3-aacc-c95ea8eafa16 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 15:47:52.095222 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:47:52.095222 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:47:52.095222 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:47:52.095222 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:47:52.095222 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:47:52.095222 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:47:52.095222 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:47:52.095222 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:47:52.095222 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:47:52.095222 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:47:52.095222 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:47:52.095222 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 15:47:52.095222 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 15:47:52.095222 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 15:47:52.096125 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 15:47:52.096125 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:47:52.096125 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:47:52.096125 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:47:52.096125 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:47:52.096125 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:47:52.096125 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:47:52.096125 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:47:52.096125 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Jun 23 15:47:52.096125 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:47:52.096125 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:47:52.096125 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:47:52.096125 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:47:52.096125 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:47:52.096125 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:47:52.096125 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:47:52.096125 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:47:52.096957 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 15:47:52.096957 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 15:47:52.096957 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 15:47:52.096957 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Jun 23 15:47:52.096957 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 257/533] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Jun 23 15:47:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:47:52.097861 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc22c7f-efab-48e3-aacc-c95ea8eafa16 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:52.100054 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32058779-4118-465f-b853-899d8d258974 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:52.100795 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32058779-4118-465f-b853-899d8d258974 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:52.101144 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32058779-4118-465f-b853-899d8d258974 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:52.101491 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32058779-4118-465f-b853-899d8d258974 None admin] PATH_INFO: `/v3/projects` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:52.102355 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32058779-4118-465f-b853-899d8d258974 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:47:52.102758 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32058779-4118-465f-b853-899d8d258974 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:52.104541 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32058779-4118-465f-b853-899d8d258974 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:52.122989 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 277/534] 10.222.0.46 () {56 vars in 1148 bytes} [Thu Jun 23 15:47:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:47:52.128845 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864ecb96-e791-4c37-80a5-adc0cc1adb0b None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:52.129141 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864ecb96-e791-4c37-80a5-adc0cc1adb0b None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:52.129424 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864ecb96-e791-4c37-80a5-adc0cc1adb0b None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:52.130733 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 278/535] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jun 23 15:47:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 15:47:52.131940 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864ecb96-e791-4c37-80a5-adc0cc1adb0b None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:52.207398 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc22c7f-efab-48e3-aacc-c95ea8eafa16 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:52.239300 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864ecb96-e791-4c37-80a5-adc0cc1adb0b None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:52.241272 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5f2e73-55c7-4505-bb1b-2c214ab7cbb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:52.241936 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5f2e73-55c7-4505-bb1b-2c214ab7cbb2 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:52.242223 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5f2e73-55c7-4505-bb1b-2c214ab7cbb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:52.242503 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5f2e73-55c7-4505-bb1b-2c214ab7cbb2 None admin] PATH_INFO: `/v3/projects` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:52.243322 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5f2e73-55c7-4505-bb1b-2c214ab7cbb2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:47:52.243635 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5f2e73-55c7-4505-bb1b-2c214ab7cbb2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:52.245082 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5f2e73-55c7-4505-bb1b-2c214ab7cbb2 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:52.258601 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 258/536] 10.222.0.46 () {56 vars in 1148 bytes} [Thu Jun 23 15:47:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:47:52.259731 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5f2e73-55c7-4505-bb1b-2c214ab7cbb2 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:52.268428 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:52.283974 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e05c0ce-e44a-47d4-9794-b136a2d49f6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:52.284645 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e05c0ce-e44a-47d4-9794-b136a2d49f6d None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:52.284931 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e05c0ce-e44a-47d4-9794-b136a2d49f6d None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:52.285211 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e05c0ce-e44a-47d4-9794-b136a2d49f6d None admin] PATH_INFO: `/v3/domains` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:52.285934 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e05c0ce-e44a-47d4-9794-b136a2d49f6d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:47:52.286248 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e05c0ce-e44a-47d4-9794-b136a2d49f6d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:52.287690 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e05c0ce-e44a-47d4-9794-b136a2d49f6d None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:52.298297 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:52.299513 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 279/537] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Jun 23 15:47:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:47:52.300872 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987ddfb3-7d55-4c8f-95d0-38e750de5257 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:52.301169 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987ddfb3-7d55-4c8f-95d0-38e750de5257 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:52.301449 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987ddfb3-7d55-4c8f-95d0-38e750de5257 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:52.318679 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-987ddfb3-7d55-4c8f-95d0-38e750de5257 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:47:52.320434 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-987ddfb3-7d55-4c8f-95d0-38e750de5257 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:47:52.381006 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5f2e73-55c7-4505-bb1b-2c214ab7cbb2 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:52.402656 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 280/538] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jun 23 15:47:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 15:47:52.402656 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987ddfb3-7d55-4c8f-95d0-38e750de5257 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:52.422593 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d07557-959e-4a33-8233-d14bba2eda78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:52.423407 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d07557-959e-4a33-8233-d14bba2eda78 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:52.423693 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d07557-959e-4a33-8233-d14bba2eda78 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:52.423970 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d07557-959e-4a33-8233-d14bba2eda78 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:52.424894 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d07557-959e-4a33-8233-d14bba2eda78 None admin] RBAC: Adding query filter params (role.id=bfa8ecadc3d5483e821420aa0f0fb41e, scope.project.id=28ecdf5b474c44098c722f677d2a0b77, user.id=ce18eaaba3a6477684fff1fd98ec8b51) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:47:52.425211 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d07557-959e-4a33-8233-d14bba2eda78 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:52.426927 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d07557-959e-4a33-8233-d14bba2eda78 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:52.442416 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 259/539] 10.222.0.46 () {56 vars in 1374 bytes} [Thu Jun 23 15:47:52 2022] GET /identity/v3/role_assignments?user.id=ce18eaaba3a6477684fff1fd98ec8b51&scope.project.id=28ecdf5b474c44098c722f677d2a0b77&role.id=bfa8ecadc3d5483e821420aa0f0fb41e => generated 259 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:47:52.443716 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d07557-959e-4a33-8233-d14bba2eda78 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:52.533632 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987ddfb3-7d55-4c8f-95d0-38e750de5257 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:52.562681 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:52.567177 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:52.568013 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d07557-959e-4a33-8233-d14bba2eda78 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:52.593388 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ddbc2f-f0a5-46e9-a6f7-a955dfd36677 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:52.593388 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ddbc2f-f0a5-46e9-a6f7-a955dfd36677 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:52.593388 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ddbc2f-f0a5-46e9-a6f7-a955dfd36677 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:52.593388 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ddbc2f-f0a5-46e9-a6f7-a955dfd36677 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:52.593776 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ddbc2f-f0a5-46e9-a6f7-a955dfd36677 None admin] RBAC: Adding query filter params (role.id=bfa8ecadc3d5483e821420aa0f0fb41e, scope.project.id=28ecdf5b474c44098c722f677d2a0b77, user.id=83970271286e451e9008bbf07cd3fa86) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:47:52.593776 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ddbc2f-f0a5-46e9-a6f7-a955dfd36677 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:52.594741 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ddbc2f-f0a5-46e9-a6f7-a955dfd36677 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:52.610719 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 281/540] 10.222.0.46 () {56 vars in 1374 bytes} [Thu Jun 23 15:47:52 2022] GET /identity/v3/role_assignments?user.id=83970271286e451e9008bbf07cd3fa86&scope.project.id=28ecdf5b474c44098c722f677d2a0b77&role.id=bfa8ecadc3d5483e821420aa0f0fb41e => generated 259 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:47:52.610719 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ddbc2f-f0a5-46e9-a6f7-a955dfd36677 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:52.641579 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1840ef20-362d-4c4e-8c3d-a19a22c75ad0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:52.642793 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1840ef20-362d-4c4e-8c3d-a19a22c75ad0 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:52.643081 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1840ef20-362d-4c4e-8c3d-a19a22c75ad0 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:52.643356 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1840ef20-362d-4c4e-8c3d-a19a22c75ad0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:52.656551 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1840ef20-362d-4c4e-8c3d-a19a22c75ad0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:52.658064 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1840ef20-362d-4c4e-8c3d-a19a22c75ad0 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:52.698213 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-1840ef20-362d-4c4e-8c3d-a19a22c75ad0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 15:47:52.698213 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:47:52.698213 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:47:52.698213 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:47:52.698213 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:47:52.698213 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:47:52.698213 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:47:52.698213 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:47:52.698213 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:47:52.698213 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:47:52.698213 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:47:52.698213 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:47:52.698213 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 15:47:52.698213 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 15:47:52.698213 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 15:47:52.699160 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 15:47:52.699160 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:47:52.699160 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:47:52.699160 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:47:52.699160 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:47:52.699160 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:47:52.699160 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:47:52.699160 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:47:52.699160 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Jun 23 15:47:52.699160 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:47:52.699160 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:47:52.699160 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:47:52.699160 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:47:52.699160 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:47:52.699160 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:47:52.699160 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:47:52.699160 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:47:52.700007 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 15:47:52.700007 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 15:47:52.700007 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 15:47:52.700007 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 15:47:52.700007 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 15:47:52.700007 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 15:47:52.700007 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 15:47:52.700007 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Jun 23 15:47:52.707463 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 260/541] 10.222.0.46 () {56 vars in 1111 bytes} [Thu Jun 23 15:47:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:47:52.708473 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1840ef20-362d-4c4e-8c3d-a19a22c75ad0 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:52.827850 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ddbc2f-f0a5-46e9-a6f7-a955dfd36677 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:52.840898 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1840ef20-362d-4c4e-8c3d-a19a22c75ad0 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:52.876709 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af796087-4dc4-4b91-9896-4bb041c7d75a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:52.877394 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af796087-4dc4-4b91-9896-4bb041c7d75a None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:52.877684 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af796087-4dc4-4b91-9896-4bb041c7d75a None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:52.877966 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af796087-4dc4-4b91-9896-4bb041c7d75a None admin] PATH_INFO: `/v3/projects` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:52.878895 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af796087-4dc4-4b91-9896-4bb041c7d75a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:47:52.879217 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af796087-4dc4-4b91-9896-4bb041c7d75a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:52.880680 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af796087-4dc4-4b91-9896-4bb041c7d75a None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:52.887428 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cfccc1-27ee-4efa-b590-7fdadff1a167 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:52.888258 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cfccc1-27ee-4efa-b590-7fdadff1a167 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:52.888545 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cfccc1-27ee-4efa-b590-7fdadff1a167 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:52.888824 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cfccc1-27ee-4efa-b590-7fdadff1a167 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:52.894058 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 261/542] 10.222.0.46 () {56 vars in 1148 bytes} [Thu Jun 23 15:47:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:47:52.900745 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af796087-4dc4-4b91-9896-4bb041c7d75a None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:52.916843 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cfccc1-27ee-4efa-b590-7fdadff1a167 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 15:47:52.928343 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:52.932971 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-13cfccc1-27ee-4efa-b590-7fdadff1a167 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 23 15:47:52.932971 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:47:52.932971 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:47:52.932971 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:47:52.932971 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:47:52.932971 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:47:52.932971 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:47:52.932971 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:47:52.932971 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:47:52.932971 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:47:52.932971 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:47:52.932971 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:47:52.932971 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 15:47:52.932971 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 15:47:52.932971 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 15:47:52.932971 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 15:47:52.933850 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 15:47:52.933850 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 15:47:52.933850 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 15:47:52.933850 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 15:47:52.933850 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:47:52.933850 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:47:52.933850 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:47:52.933850 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:47:52.933850 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:47:52.933850 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:47:52.933850 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:47:52.933850 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Jun 23 15:47:52.933850 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:47:52.933850 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:47:52.933850 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:47:52.933850 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:47:52.934638 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:47:52.934638 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:47:52.934638 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:47:52.934638 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:47:52.934638 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 15:47:52.934638 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 15:47:52.934638 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 15:47:52.934638 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 15:47:52.934638 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 15:47:52.934638 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 15:47:52.934638 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 23 15:47:52.934638 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Jun 23 15:47:52.934638 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 282/543] 10.222.0.46 () {56 vars in 1102 bytes} [Thu Jun 23 15:47:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 329 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:47:52.943711 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cfccc1-27ee-4efa-b590-7fdadff1a167 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:52.989838 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:53.011729 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af796087-4dc4-4b91-9896-4bb041c7d75a None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:53.049659 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9779f2b1-27c0-4ecb-bddc-c0a860a7633d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:53.050332 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9779f2b1-27c0-4ecb-bddc-c0a860a7633d None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:53.050618 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9779f2b1-27c0-4ecb-bddc-c0a860a7633d None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:53.050922 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9779f2b1-27c0-4ecb-bddc-c0a860a7633d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:53.051767 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9779f2b1-27c0-4ecb-bddc-c0a860a7633d None admin] RBAC: Adding query filter params (role.id=bfa8ecadc3d5483e821420aa0f0fb41e, scope.project.id=28ecdf5b474c44098c722f677d2a0b77, user.id=51784064951a4619ae09b9167fb0fd0d) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:47:53.052078 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9779f2b1-27c0-4ecb-bddc-c0a860a7633d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:53.053660 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9779f2b1-27c0-4ecb-bddc-c0a860a7633d None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:53.060200 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 262/544] 10.222.0.46 () {56 vars in 1374 bytes} [Thu Jun 23 15:47:52 2022] GET /identity/v3/role_assignments?user.id=51784064951a4619ae09b9167fb0fd0d&scope.project.id=28ecdf5b474c44098c722f677d2a0b77&role.id=bfa8ecadc3d5483e821420aa0f0fb41e => generated 259 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 15:47:53.065521 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cfccc1-27ee-4efa-b590-7fdadff1a167 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:53.067511 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda7bb18-2abb-4d4d-85d6-90a093c676fb None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:53.067846 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda7bb18-2abb-4d4d-85d6-90a093c676fb None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:53.068163 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda7bb18-2abb-4d4d-85d6-90a093c676fb None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:53.070068 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 263/545] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jun 23 15:47:53 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 15:47:53.082660 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad68084-a13e-4529-8efd-e960f3164419 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:53.082979 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad68084-a13e-4529-8efd-e960f3164419 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:53.083248 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad68084-a13e-4529-8efd-e960f3164419 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:53.119030 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aca8961-d6a9-496e-b741-f95ef8bc76b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:53.119749 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aca8961-d6a9-496e-b741-f95ef8bc76b2 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:53.120042 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aca8961-d6a9-496e-b741-f95ef8bc76b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:53.120320 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aca8961-d6a9-496e-b741-f95ef8bc76b2 None admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:53.121102 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aca8961-d6a9-496e-b741-f95ef8bc76b2 None admin] RBAC: Adding query filter params (name=service) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:47:53.121413 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aca8961-d6a9-496e-b741-f95ef8bc76b2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:53.122879 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aca8961-d6a9-496e-b741-f95ef8bc76b2 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:53.125390 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-fad68084-a13e-4529-8efd-e960f3164419 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:47:53.126640 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-fad68084-a13e-4529-8efd-e960f3164419 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:47:53.140047 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 283/546] 10.222.0.46 () {56 vars in 1103 bytes} [Thu Jun 23 15:47:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:47:53.145996 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aca8961-d6a9-496e-b741-f95ef8bc76b2 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:53.263172 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 264/547] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jun 23 15:47:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 15:47:53.284046 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad68084-a13e-4529-8efd-e960f3164419 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:53.302385 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:53.309170 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:53.315892 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aca8961-d6a9-496e-b741-f95ef8bc76b2 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:53.354806 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79a0102-aab3-4c0b-a0d8-1a38d04ca457 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:53.356421 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79a0102-aab3-4c0b-a0d8-1a38d04ca457 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:53.356707 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79a0102-aab3-4c0b-a0d8-1a38d04ca457 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:53.356985 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79a0102-aab3-4c0b-a0d8-1a38d04ca457 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:53.368964 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79a0102-aab3-4c0b-a0d8-1a38d04ca457 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:53.370816 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79a0102-aab3-4c0b-a0d8-1a38d04ca457 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:53.389453 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-d79a0102-aab3-4c0b-a0d8-1a38d04ca457 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 15:47:53.389453 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:47:53.389453 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:47:53.389453 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:47:53.389453 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:47:53.389453 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:47:53.389453 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:47:53.389453 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:47:53.389453 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:47:53.389453 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:47:53.389453 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:47:53.389453 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:47:53.389453 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 15:47:53.389453 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 15:47:53.389453 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 15:47:53.390332 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 15:47:53.390332 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:47:53.390332 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:47:53.390332 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:47:53.390332 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:47:53.390332 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:47:53.390332 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:47:53.390332 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:47:53.390332 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Jun 23 15:47:53.390332 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:47:53.390332 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:47:53.390332 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:47:53.390332 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:47:53.390332 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:47:53.390332 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:47:53.390332 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:47:53.390332 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:47:53.391190 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 15:47:53.391190 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 15:47:53.391190 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 15:47:53.391190 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Jun 23 15:47:53.391190 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 284/548] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Jun 23 15:47:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:47:53.401949 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79a0102-aab3-4c0b-a0d8-1a38d04ca457 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:53.495016 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad68084-a13e-4529-8efd-e960f3164419 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:53.575244 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7e6f57-7a96-4eb1-8af1-429cda9bcb69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:53.576119 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7e6f57-7a96-4eb1-8af1-429cda9bcb69 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:53.576340 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7e6f57-7a96-4eb1-8af1-429cda9bcb69 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:53.576613 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7e6f57-7a96-4eb1-8af1-429cda9bcb69 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:53.596554 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7e6f57-7a96-4eb1-8af1-429cda9bcb69 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 15:47:53.613429 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79a0102-aab3-4c0b-a0d8-1a38d04ca457 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:53.614967 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-7c7e6f57-7a96-4eb1-8af1-429cda9bcb69 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 23 15:47:53.614967 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:47:53.614967 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:47:53.614967 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:47:53.614967 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:47:53.614967 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:47:53.614967 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:47:53.614967 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:47:53.614967 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:47:53.614967 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:47:53.614967 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:47:53.614967 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:47:53.614967 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 15:47:53.614967 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 15:47:53.614967 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 15:47:53.614967 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 15:47:53.616368 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 15:47:53.616368 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 15:47:53.616368 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 15:47:53.616368 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 15:47:53.616368 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:47:53.616368 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:47:53.616368 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:47:53.616368 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:47:53.616368 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:47:53.616368 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:47:53.616368 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:47:53.616368 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Jun 23 15:47:53.616368 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:47:53.616368 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:47:53.616368 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:47:53.616368 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:47:53.617659 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:47:53.617659 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:47:53.617659 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:47:53.617659 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:47:53.617659 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 15:47:53.617659 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 15:47:53.617659 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 15:47:53.617659 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 15:47:53.617659 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 15:47:53.617659 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 15:47:53.617659 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 23 15:47:53.617659 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Jun 23 15:47:53.623561 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 265/549] 10.222.0.46 () {56 vars in 1102 bytes} [Thu Jun 23 15:47:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 340 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:47:53.624488 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7e6f57-7a96-4eb1-8af1-429cda9bcb69 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:53.686951 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d419343-0422-4f3a-a296-95aae29e1b54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:53.687629 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d419343-0422-4f3a-a296-95aae29e1b54 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:53.687914 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d419343-0422-4f3a-a296-95aae29e1b54 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:53.688186 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d419343-0422-4f3a-a296-95aae29e1b54 None admin] PATH_INFO: `/v3/domains` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:53.688913 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d419343-0422-4f3a-a296-95aae29e1b54 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:47:53.689218 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d419343-0422-4f3a-a296-95aae29e1b54 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:53.691759 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d419343-0422-4f3a-a296-95aae29e1b54 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:53.716089 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 285/550] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Jun 23 15:47:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:47:53.720363 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d419343-0422-4f3a-a296-95aae29e1b54 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:53.743039 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:53.754557 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:53.784793 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7e6f57-7a96-4eb1-8af1-429cda9bcb69 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:53.854924 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c168df-2625-4d48-9c5b-be68137f6513 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:53.855973 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c168df-2625-4d48-9c5b-be68137f6513 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:53.856452 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c168df-2625-4d48-9c5b-be68137f6513 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:53.856958 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c168df-2625-4d48-9c5b-be68137f6513 None admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:53.857901 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c168df-2625-4d48-9c5b-be68137f6513 None admin] RBAC: Adding query filter params (name=service) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:47:53.859790 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c168df-2625-4d48-9c5b-be68137f6513 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:53.861365 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c168df-2625-4d48-9c5b-be68137f6513 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:53.879116 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 266/551] 10.222.0.46 () {56 vars in 1103 bytes} [Thu Jun 23 15:47:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:47:53.885290 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c168df-2625-4d48-9c5b-be68137f6513 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:53.911706 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d419343-0422-4f3a-a296-95aae29e1b54 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:53.947751 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57e310c-af46-4917-acea-6dcf1bef955d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:53.948555 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57e310c-af46-4917-acea-6dcf1bef955d None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:53.948840 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57e310c-af46-4917-acea-6dcf1bef955d None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:53.949115 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57e310c-af46-4917-acea-6dcf1bef955d None admin] PATH_INFO: `/v3/users/swift` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:53.956690 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57e310c-af46-4917-acea-6dcf1bef955d None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:53.958163 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57e310c-af46-4917-acea-6dcf1bef955d None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:53.986799 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-d57e310c-af46-4917-acea-6dcf1bef955d None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 23 15:47:53.986799 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:47:53.986799 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:47:53.986799 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:47:53.986799 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:47:53.986799 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:47:53.986799 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:47:53.986799 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:47:53.986799 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:47:53.986799 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:47:53.986799 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:47:53.986799 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:47:53.986799 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 15:47:53.986799 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 15:47:53.986799 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 15:47:53.986799 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 15:47:53.987847 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:47:53.987847 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:47:53.987847 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 15:47:53.987847 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 15:47:53.987847 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 15:47:53.987847 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 15:47:53.987847 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:47:53.987847 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:47:53.987847 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:47:53.987847 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:47:53.987847 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:47:53.987847 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Jun 23 15:47:53.987847 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:47:53.987847 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:47:53.987847 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:47:53.987847 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:47:53.987847 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:47:53.987847 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:47:53.988833 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:47:53.988833 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:47:53.988833 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 15:47:53.988833 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 15:47:53.988833 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 15:47:53.988833 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 15:47:53.988833 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 15:47:53.988833 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 15:47:53.988833 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 23 15:47:53.988833 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Jun 23 15:47:53.988833 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 286/552] 10.222.0.46 () {56 vars in 1096 bytes} [Thu Jun 23 15:47:53 2022] GET /identity/v3/users/swift => generated 83 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:47:53.995638 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57e310c-af46-4917-acea-6dcf1bef955d None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:54.072608 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c168df-2625-4d48-9c5b-be68137f6513 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:54.088039 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:54.133181 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57e310c-af46-4917-acea-6dcf1bef955d None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:54.148602 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb3b617-6fba-474f-8db8-f5b8ecd9587e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:54.149730 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb3b617-6fba-474f-8db8-f5b8ecd9587e None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:54.150459 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb3b617-6fba-474f-8db8-f5b8ecd9587e None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:54.151088 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb3b617-6fba-474f-8db8-f5b8ecd9587e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:54.154206 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:54.165943 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb3b617-6fba-474f-8db8-f5b8ecd9587e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:54.167729 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb3b617-6fba-474f-8db8-f5b8ecd9587e None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:54.187667 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-3eb3b617-6fba-474f-8db8-f5b8ecd9587e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 15:47:54.187667 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:47:54.187667 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:47:54.187667 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:47:54.187667 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:47:54.187667 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:47:54.187667 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:47:54.187667 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:47:54.187667 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:47:54.187667 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:47:54.187667 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:47:54.187667 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:47:54.187667 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 15:47:54.187667 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 15:47:54.187667 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 15:47:54.188557 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 15:47:54.188557 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:47:54.188557 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:47:54.188557 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:47:54.188557 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:47:54.188557 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:47:54.188557 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:47:54.188557 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:47:54.188557 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Jun 23 15:47:54.188557 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:47:54.188557 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:47:54.188557 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:47:54.188557 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:47:54.188557 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:47:54.188557 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:47:54.188557 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:47:54.188557 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:47:54.189422 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 15:47:54.189422 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 15:47:54.189422 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 15:47:54.189422 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Jun 23 15:47:54.191839 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 267/553] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Jun 23 15:47:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 15:47:54.201289 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb3b617-6fba-474f-8db8-f5b8ecd9587e None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:54.219869 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8c5e5a-1439-43cc-aa19-98d7c2593dfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:54.220549 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8c5e5a-1439-43cc-aa19-98d7c2593dfd None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:54.220839 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8c5e5a-1439-43cc-aa19-98d7c2593dfd None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:54.221114 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8c5e5a-1439-43cc-aa19-98d7c2593dfd None admin] PATH_INFO: `/v3/users` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:54.221968 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8c5e5a-1439-43cc-aa19-98d7c2593dfd None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:47:54.222278 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8c5e5a-1439-43cc-aa19-98d7c2593dfd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:54.225138 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8c5e5a-1439-43cc-aa19-98d7c2593dfd None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:54.257150 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 287/554] 10.222.0.46 () {56 vars in 1135 bytes} [Thu Jun 23 15:47:53 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:47:54.260746 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8c5e5a-1439-43cc-aa19-98d7c2593dfd None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:54.368658 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb3b617-6fba-474f-8db8-f5b8ecd9587e None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:54.407279 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fcad53-67b0-4484-81fd-688aec86dc15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:54.407882 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fcad53-67b0-4484-81fd-688aec86dc15 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:54.408166 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fcad53-67b0-4484-81fd-688aec86dc15 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:54.408443 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fcad53-67b0-4484-81fd-688aec86dc15 None admin] PATH_INFO: `/v3/domains` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:54.409201 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fcad53-67b0-4484-81fd-688aec86dc15 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:47:54.409490 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fcad53-67b0-4484-81fd-688aec86dc15 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:54.410999 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fcad53-67b0-4484-81fd-688aec86dc15 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:54.431024 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 268/555] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Jun 23 15:47:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:47:54.432448 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fcad53-67b0-4484-81fd-688aec86dc15 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:54.494352 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:54.540726 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8c5e5a-1439-43cc-aa19-98d7c2593dfd None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:54.554519 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fcad53-67b0-4484-81fd-688aec86dc15 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:54.600462 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e979067-5721-428b-baf6-51014486f433 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:54.602165 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e979067-5721-428b-baf6-51014486f433 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:54.603106 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e979067-5721-428b-baf6-51014486f433 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:54.603401 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e979067-5721-428b-baf6-51014486f433 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:54.624524 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e979067-5721-428b-baf6-51014486f433 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:54.625979 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e979067-5721-428b-baf6-51014486f433 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:54.626655 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:54.644197 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2253cde-99a6-4c26-9a63-9e2c9fc26871 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:54.644912 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2253cde-99a6-4c26-9a63-9e2c9fc26871 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:54.645202 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2253cde-99a6-4c26-9a63-9e2c9fc26871 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:54.645481 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2253cde-99a6-4c26-9a63-9e2c9fc26871 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:54.652648 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-1e979067-5721-428b-baf6-51014486f433 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 23 15:47:54.652648 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:47:54.652648 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:47:54.652648 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:47:54.652648 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:47:54.652648 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:47:54.652648 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:47:54.652648 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:47:54.652648 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:47:54.652648 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:47:54.652648 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:47:54.652648 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:47:54.652648 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 15:47:54.652648 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 15:47:54.652648 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 15:47:54.652648 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 15:47:54.654043 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:47:54.654043 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:47:54.654043 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 15:47:54.654043 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 15:47:54.654043 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 15:47:54.654043 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 15:47:54.654043 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:47:54.654043 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:47:54.654043 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:47:54.654043 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:47:54.654043 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:47:54.654043 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Jun 23 15:47:54.654043 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:47:54.654043 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:47:54.654043 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:47:54.654043 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:47:54.654043 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:47:54.654043 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:47:54.655382 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:47:54.655382 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:47:54.655382 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 15:47:54.655382 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 15:47:54.655382 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 15:47:54.655382 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 15:47:54.655382 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 15:47:54.655382 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 15:47:54.655382 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 23 15:47:54.655382 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Jun 23 15:47:54.655382 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 269/556] 10.222.0.46 () {56 vars in 1093 bytes} [Thu Jun 23 15:47:54 2022] GET /identity/v3/users/nova => generated 82 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 15:47:54.667031 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e979067-5721-428b-baf6-51014486f433 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:54.748363 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2253cde-99a6-4c26-9a63-9e2c9fc26871 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:54.750067 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2253cde-99a6-4c26-9a63-9e2c9fc26871 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:54.873000 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e979067-5721-428b-baf6-51014486f433 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:54.884234 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 288/557] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Jun 23 15:47:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 624 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 15:47:54.888862 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:54.889019 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658e1037-4e20-4bf2-a668-2e54d9829d2a None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:54.889019 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658e1037-4e20-4bf2-a668-2e54d9829d2a None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:54.889019 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658e1037-4e20-4bf2-a668-2e54d9829d2a None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:54.890988 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 289/558] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jun 23 15:47:54 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 15:47:54.893068 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ce3f4a-9c09-4fef-9641-d23e8b9a7484 None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:54.893448 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ce3f4a-9c09-4fef-9641-d23e8b9a7484 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:54.893725 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ce3f4a-9c09-4fef-9641-d23e8b9a7484 None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:54.895147 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 290/559] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jun 23 15:47:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 15:47:54.895707 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ce3f4a-9c09-4fef-9641-d23e8b9a7484 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:54.920900 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133b8436-7c5e-49dc-b70b-0781893d31d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:54.921531 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133b8436-7c5e-49dc-b70b-0781893d31d3 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:54.921817 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133b8436-7c5e-49dc-b70b-0781893d31d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:54.922095 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133b8436-7c5e-49dc-b70b-0781893d31d3 None admin] PATH_INFO: `/v3/users` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:54.922973 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133b8436-7c5e-49dc-b70b-0781893d31d3 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:47:54.923286 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133b8436-7c5e-49dc-b70b-0781893d31d3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:54.924726 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133b8436-7c5e-49dc-b70b-0781893d31d3 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:54.929203 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:54.944716 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 270/560] 10.222.0.46 () {56 vars in 1133 bytes} [Thu Jun 23 15:47:54 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:47:54.946096 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4180c44e-77e8-4562-91c5-b81ab40e1a8b None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:54.946393 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4180c44e-77e8-4562-91c5-b81ab40e1a8b None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:54.946676 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4180c44e-77e8-4562-91c5-b81ab40e1a8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:54.966353 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-4180c44e-77e8-4562-91c5-b81ab40e1a8b None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:47:54.967575 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-4180c44e-77e8-4562-91c5-b81ab40e1a8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:47:55.015265 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ce3f4a-9c09-4fef-9641-d23e8b9a7484 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:55.041419 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 271/561] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jun 23 15:47:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 15:47:55.043035 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afc7b33-d0bc-4146-958b-119d167f5bc7 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:55.043635 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afc7b33-d0bc-4146-958b-119d167f5bc7 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:55.044082 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afc7b33-d0bc-4146-958b-119d167f5bc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:55.063449 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-9afc7b33-d0bc-4146-958b-119d167f5bc7 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:47:55.065040 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-9afc7b33-d0bc-4146-958b-119d167f5bc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:47:55.069255 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f2a321-a9c3-4ae9-a264-ed35f0e80558 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:55.070064 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f2a321-a9c3-4ae9-a264-ed35f0e80558 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:55.070350 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f2a321-a9c3-4ae9-a264-ed35f0e80558 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:55.070631 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f2a321-a9c3-4ae9-a264-ed35f0e80558 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:55.083042 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f2a321-a9c3-4ae9-a264-ed35f0e80558 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:55.085036 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f2a321-a9c3-4ae9-a264-ed35f0e80558 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:55.105073 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-79f2a321-a9c3-4ae9-a264-ed35f0e80558 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 15:47:55.105073 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:47:55.105073 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:47:55.105073 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:47:55.105073 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:47:55.105073 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:47:55.105073 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:47:55.105073 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:47:55.105073 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:47:55.105073 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:47:55.105073 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:47:55.105073 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:47:55.105073 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 15:47:55.105073 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 15:47:55.105073 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 15:47:55.106216 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 15:47:55.106216 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:47:55.106216 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:47:55.106216 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:47:55.106216 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:47:55.106216 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:47:55.106216 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:47:55.106216 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:47:55.106216 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Jun 23 15:47:55.106216 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:47:55.106216 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:47:55.106216 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:47:55.106216 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:47:55.106216 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:47:55.106216 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:47:55.106216 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:47:55.106216 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:47:55.107380 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 15:47:55.107380 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 15:47:55.107380 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 15:47:55.107380 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Jun 23 15:47:55.108791 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 291/562] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Jun 23 15:47:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:47:55.112091 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f2a321-a9c3-4ae9-a264-ed35f0e80558 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:55.137573 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 272/563] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jun 23 15:47:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 15:47:55.139611 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afc7b33-d0bc-4146-958b-119d167f5bc7 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:55.180765 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:55.204851 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:55.229882 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f2a321-a9c3-4ae9-a264-ed35f0e80558 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:55.261412 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afc7b33-d0bc-4146-958b-119d167f5bc7 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:55.267462 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae21f389-4c12-4adf-888e-a47a677fff17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:55.268253 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae21f389-4c12-4adf-888e-a47a677fff17 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:55.268641 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae21f389-4c12-4adf-888e-a47a677fff17 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:55.268824 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae21f389-4c12-4adf-888e-a47a677fff17 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:55.280647 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae21f389-4c12-4adf-888e-a47a677fff17 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:55.282133 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae21f389-4c12-4adf-888e-a47a677fff17 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:55.297497 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d27a44-51ca-4a32-bdd9-6c77c30b0688 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:55.298421 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d27a44-51ca-4a32-bdd9-6c77c30b0688 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:55.298600 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d27a44-51ca-4a32-bdd9-6c77c30b0688 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:55.298896 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d27a44-51ca-4a32-bdd9-6c77c30b0688 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:55.300880 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-ae21f389-4c12-4adf-888e-a47a677fff17 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 15:47:55.300880 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:47:55.300880 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:47:55.300880 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:47:55.300880 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:47:55.300880 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:47:55.300880 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:47:55.300880 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:47:55.300880 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:47:55.300880 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:47:55.300880 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:47:55.300880 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:47:55.300880 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 15:47:55.300880 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 15:47:55.300880 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 15:47:55.301710 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 15:47:55.301710 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:47:55.301710 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:47:55.301710 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:47:55.301710 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:47:55.301710 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:47:55.301710 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:47:55.301710 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:47:55.301710 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Jun 23 15:47:55.301710 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:47:55.301710 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:47:55.301710 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:47:55.301710 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:47:55.301710 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:47:55.301710 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:47:55.301710 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:47:55.301710 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:47:55.302603 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 15:47:55.302603 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 15:47:55.302603 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 15:47:55.302603 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Jun 23 15:47:55.302603 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 292/564] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Jun 23 15:47:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:47:55.302962 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae21f389-4c12-4adf-888e-a47a677fff17 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:55.314920 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d27a44-51ca-4a32-bdd9-6c77c30b0688 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 15:47:55.323179 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-10d27a44-51ca-4a32-bdd9-6c77c30b0688 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 23 15:47:55.323179 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:47:55.323179 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:47:55.323179 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:47:55.323179 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:47:55.323179 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:47:55.323179 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:47:55.323179 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:47:55.323179 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:47:55.323179 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:47:55.323179 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:47:55.323179 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:47:55.323179 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 15:47:55.323179 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 15:47:55.323179 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 15:47:55.323179 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 15:47:55.324092 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 15:47:55.324092 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 15:47:55.324092 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 15:47:55.324092 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 15:47:55.324092 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:47:55.324092 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:47:55.324092 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:47:55.324092 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:47:55.324092 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:47:55.324092 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:47:55.324092 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:47:55.324092 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Jun 23 15:47:55.324092 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:47:55.324092 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:47:55.324092 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:47:55.324092 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:47:55.324923 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:47:55.324923 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:47:55.324923 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:47:55.324923 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:47:55.324923 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 15:47:55.324923 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 15:47:55.324923 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 15:47:55.324923 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 15:47:55.324923 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 15:47:55.324923 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 15:47:55.324923 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 23 15:47:55.324923 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Jun 23 15:47:55.324923 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 273/565] 10.222.0.46 () {56 vars in 1102 bytes} [Thu Jun 23 15:47:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:47:55.325605 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d27a44-51ca-4a32-bdd9-6c77c30b0688 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:55.410404 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae21f389-4c12-4adf-888e-a47a677fff17 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:55.418488 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:55.435354 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d27a44-51ca-4a32-bdd9-6c77c30b0688 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:55.440131 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:55.446903 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74be6a4e-d911-4f87-bf62-880a5634c213 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:55.447579 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74be6a4e-d911-4f87-bf62-880a5634c213 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:55.447869 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74be6a4e-d911-4f87-bf62-880a5634c213 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:55.448153 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74be6a4e-d911-4f87-bf62-880a5634c213 None admin] PATH_INFO: `/v3/domains` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:55.448881 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74be6a4e-d911-4f87-bf62-880a5634c213 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:47:55.449191 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74be6a4e-d911-4f87-bf62-880a5634c213 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:55.450568 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74be6a4e-d911-4f87-bf62-880a5634c213 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:55.462430 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 293/566] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Jun 23 15:47:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:47:55.463281 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74be6a4e-d911-4f87-bf62-880a5634c213 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:55.471813 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a7a5ac-1fbe-43ee-aa93-f75115228f74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:55.472640 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a7a5ac-1fbe-43ee-aa93-f75115228f74 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:55.472924 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a7a5ac-1fbe-43ee-aa93-f75115228f74 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:55.473199 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a7a5ac-1fbe-43ee-aa93-f75115228f74 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:55.483240 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a7a5ac-1fbe-43ee-aa93-f75115228f74 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 15:47:55.490962 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-b3a7a5ac-1fbe-43ee-aa93-f75115228f74 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 23 15:47:55.490962 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:47:55.490962 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:47:55.490962 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:47:55.490962 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:47:55.490962 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:47:55.490962 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:47:55.490962 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:47:55.490962 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:47:55.490962 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:47:55.490962 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:47:55.490962 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:47:55.490962 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 15:47:55.490962 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 15:47:55.490962 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 15:47:55.490962 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 15:47:55.491929 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 15:47:55.491929 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 15:47:55.491929 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 15:47:55.491929 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 15:47:55.491929 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:47:55.491929 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:47:55.491929 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:47:55.491929 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:47:55.491929 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:47:55.491929 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:47:55.491929 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:47:55.491929 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Jun 23 15:47:55.491929 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:47:55.491929 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:47:55.491929 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:47:55.491929 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:47:55.492756 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:47:55.492756 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:47:55.492756 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:47:55.492756 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:47:55.492756 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 15:47:55.492756 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 15:47:55.492756 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 15:47:55.492756 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 15:47:55.492756 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 15:47:55.492756 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 15:47:55.492756 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 23 15:47:55.492756 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Jun 23 15:47:55.492756 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 274/567] 10.222.0.46 () {56 vars in 1102 bytes} [Thu Jun 23 15:47:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:47:55.493448 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a7a5ac-1fbe-43ee-aa93-f75115228f74 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:55.572809 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74be6a4e-d911-4f87-bf62-880a5634c213 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:55.596963 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a7a5ac-1fbe-43ee-aa93-f75115228f74 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:55.606138 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fc067b-2083-42f4-8197-11dc8cd62618 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:55.606852 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fc067b-2083-42f4-8197-11dc8cd62618 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:55.607138 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fc067b-2083-42f4-8197-11dc8cd62618 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:55.607415 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fc067b-2083-42f4-8197-11dc8cd62618 None admin] PATH_INFO: `/v3/domains` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:55.608138 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fc067b-2083-42f4-8197-11dc8cd62618 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:47:55.608447 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fc067b-2083-42f4-8197-11dc8cd62618 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:55.609826 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fc067b-2083-42f4-8197-11dc8cd62618 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:55.621407 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 294/568] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Jun 23 15:47:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:47:55.622753 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fd1090-2f20-4933-9a20-bd5489713444 None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:55.623043 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fd1090-2f20-4933-9a20-bd5489713444 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:55.623320 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fd1090-2f20-4933-9a20-bd5489713444 None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:55.624542 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 295/569] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jun 23 15:47:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 15:47:55.625237 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fd1090-2f20-4933-9a20-bd5489713444 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:55.636466 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce89aeda-10ef-4de8-a006-39fd23d9467b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:55.637155 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce89aeda-10ef-4de8-a006-39fd23d9467b None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:55.637441 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce89aeda-10ef-4de8-a006-39fd23d9467b None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:55.637722 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce89aeda-10ef-4de8-a006-39fd23d9467b None admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:55.638480 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce89aeda-10ef-4de8-a006-39fd23d9467b None admin] RBAC: Adding query filter params (name=service) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:47:55.638822 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce89aeda-10ef-4de8-a006-39fd23d9467b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:55.640224 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce89aeda-10ef-4de8-a006-39fd23d9467b None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:55.648970 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 275/570] 10.222.0.46 () {56 vars in 1103 bytes} [Thu Jun 23 15:47:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:47:55.649763 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce89aeda-10ef-4de8-a006-39fd23d9467b None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:55.662121 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:55.678058 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:55.731053 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fd1090-2f20-4933-9a20-bd5489713444 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:55.750529 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce89aeda-10ef-4de8-a006-39fd23d9467b None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:55.764894 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe00698-72f7-477c-bc50-44446cc46500 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:55.765694 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe00698-72f7-477c-bc50-44446cc46500 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:55.765976 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe00698-72f7-477c-bc50-44446cc46500 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:55.766250 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe00698-72f7-477c-bc50-44446cc46500 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:55.772931 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe00698-72f7-477c-bc50-44446cc46500 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:55.774390 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe00698-72f7-477c-bc50-44446cc46500 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:55.784015 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d55b7f-d4f6-4c36-a5cf-0d7336eb47fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:55.784683 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d55b7f-d4f6-4c36-a5cf-0d7336eb47fc None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:55.784967 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d55b7f-d4f6-4c36-a5cf-0d7336eb47fc None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:55.785246 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d55b7f-d4f6-4c36-a5cf-0d7336eb47fc None admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:55.785994 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d55b7f-d4f6-4c36-a5cf-0d7336eb47fc None admin] RBAC: Adding query filter params (name=service) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:47:55.786305 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d55b7f-d4f6-4c36-a5cf-0d7336eb47fc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:55.787747 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d55b7f-d4f6-4c36-a5cf-0d7336eb47fc None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:55.789710 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-7fe00698-72f7-477c-bc50-44446cc46500 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 15:47:55.789710 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:47:55.789710 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:47:55.789710 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:47:55.789710 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:47:55.789710 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:47:55.789710 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:47:55.789710 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:47:55.789710 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:47:55.789710 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:47:55.789710 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:47:55.789710 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:47:55.789710 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 15:47:55.789710 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 15:47:55.789710 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 15:47:55.790725 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 15:47:55.790725 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:47:55.790725 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:47:55.790725 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:47:55.790725 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:47:55.790725 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:47:55.790725 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:47:55.790725 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:47:55.790725 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Jun 23 15:47:55.790725 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:47:55.790725 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:47:55.790725 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:47:55.790725 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:47:55.790725 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:47:55.790725 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:47:55.790725 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:47:55.790725 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:47:55.791658 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 15:47:55.791658 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 15:47:55.791658 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 15:47:55.791658 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 15:47:55.791658 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 15:47:55.791658 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 15:47:55.791658 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 15:47:55.791658 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Jun 23 15:47:55.791658 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 296/571] 10.222.0.46 () {56 vars in 1111 bytes} [Thu Jun 23 15:47:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:47:55.792134 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe00698-72f7-477c-bc50-44446cc46500 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:55.802719 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 276/572] 10.222.0.46 () {56 vars in 1103 bytes} [Thu Jun 23 15:47:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:47:55.802719 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2f9412-e368-4c0d-b4f0-af2ee38c7d94 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:55.802719 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2f9412-e368-4c0d-b4f0-af2ee38c7d94 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:55.802719 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2f9412-e368-4c0d-b4f0-af2ee38c7d94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:55.816871 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-7a2f9412-e368-4c0d-b4f0-af2ee38c7d94 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:47:55.818056 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-7a2f9412-e368-4c0d-b4f0-af2ee38c7d94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:47:55.878617 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:55.883661 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 277/573] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jun 23 15:47:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 15:47:55.884673 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2f9412-e368-4c0d-b4f0-af2ee38c7d94 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:55.899954 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe00698-72f7-477c-bc50-44446cc46500 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:55.910829 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:55.934534 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6565d3-892d-432e-809a-a8878459ea38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:55.935368 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6565d3-892d-432e-809a-a8878459ea38 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:55.935659 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6565d3-892d-432e-809a-a8878459ea38 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:55.935937 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6565d3-892d-432e-809a-a8878459ea38 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:55.942588 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6565d3-892d-432e-809a-a8878459ea38 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:55.944645 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6565d3-892d-432e-809a-a8878459ea38 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:55.959263 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-cd6565d3-892d-432e-809a-a8878459ea38 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 15:47:55.959263 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:47:55.959263 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:47:55.959263 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:47:55.959263 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:47:55.959263 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:47:55.959263 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:47:55.959263 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:47:55.959263 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:47:55.959263 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:47:55.959263 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:47:55.959263 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:47:55.959263 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 15:47:55.959263 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 15:47:55.959263 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 15:47:55.960233 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 15:47:55.960233 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:47:55.960233 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:47:55.960233 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:47:55.960233 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:47:55.960233 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:47:55.960233 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:47:55.960233 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:47:55.960233 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Jun 23 15:47:55.960233 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:47:55.960233 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:47:55.960233 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:47:55.960233 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:47:55.960233 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:47:55.960233 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:47:55.960233 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:47:55.960233 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:47:55.961128 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 15:47:55.961128 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 15:47:55.961128 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 15:47:55.961128 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 15:47:55.961128 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 15:47:55.961128 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 15:47:55.961128 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 15:47:55.961128 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Jun 23 15:47:55.961128 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 297/574] 10.222.0.46 () {56 vars in 1111 bytes} [Thu Jun 23 15:47:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:47:55.961610 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6565d3-892d-432e-809a-a8878459ea38 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:55.996170 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2f9412-e368-4c0d-b4f0-af2ee38c7d94 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:56.029834 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffe45b7-8329-49fa-8465-01e27866d0ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:56.030629 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffe45b7-8329-49fa-8465-01e27866d0ca None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:56.030946 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffe45b7-8329-49fa-8465-01e27866d0ca None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:56.031221 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffe45b7-8329-49fa-8465-01e27866d0ca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:56.041199 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffe45b7-8329-49fa-8465-01e27866d0ca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:56.042677 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffe45b7-8329-49fa-8465-01e27866d0ca None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:56.060325 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6565d3-892d-432e-809a-a8878459ea38 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:56.060581 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-5ffe45b7-8329-49fa-8465-01e27866d0ca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 15:47:56.060581 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:47:56.060581 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:47:56.060581 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:47:56.060581 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:47:56.060581 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:47:56.060581 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:47:56.060581 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:47:56.060581 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:47:56.060581 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:47:56.060581 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:47:56.060581 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:47:56.060581 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 15:47:56.060581 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 15:47:56.060581 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 15:47:56.061563 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 15:47:56.061563 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:47:56.061563 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:47:56.061563 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:47:56.061563 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:47:56.061563 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:47:56.061563 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:47:56.061563 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:47:56.061563 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Jun 23 15:47:56.061563 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:47:56.061563 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:47:56.061563 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:47:56.061563 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:47:56.061563 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:47:56.061563 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:47:56.061563 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:47:56.061563 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:47:56.062626 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 15:47:56.062626 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 15:47:56.062626 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 15:47:56.062626 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Jun 23 15:47:56.062626 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 278/575] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Jun 23 15:47:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:47:56.062626 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffe45b7-8329-49fa-8465-01e27866d0ca None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:56.099785 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5305599-a831-4995-8996-241e743a6553 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:56.100480 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5305599-a831-4995-8996-241e743a6553 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:56.100764 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5305599-a831-4995-8996-241e743a6553 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:56.101043 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5305599-a831-4995-8996-241e743a6553 None admin] PATH_INFO: `/v3/projects` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:56.101832 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5305599-a831-4995-8996-241e743a6553 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:47:56.102140 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5305599-a831-4995-8996-241e743a6553 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:56.103641 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5305599-a831-4995-8996-241e743a6553 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:56.120340 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 298/576] 10.222.0.46 () {56 vars in 1148 bytes} [Thu Jun 23 15:47:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:47:56.120340 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5305599-a831-4995-8996-241e743a6553 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:56.120605 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:56.149120 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:56.164868 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffe45b7-8329-49fa-8465-01e27866d0ca None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:56.198768 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697a9d26-4c02-49b7-bfcc-d43dc3e425df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:56.199564 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697a9d26-4c02-49b7-bfcc-d43dc3e425df None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:56.199845 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697a9d26-4c02-49b7-bfcc-d43dc3e425df None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:56.200123 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697a9d26-4c02-49b7-bfcc-d43dc3e425df None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:56.210082 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697a9d26-4c02-49b7-bfcc-d43dc3e425df None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:56.211573 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697a9d26-4c02-49b7-bfcc-d43dc3e425df None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:56.217922 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5305599-a831-4995-8996-241e743a6553 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:56.229521 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-697a9d26-4c02-49b7-bfcc-d43dc3e425df None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 15:47:56.229521 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:47:56.229521 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:47:56.229521 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:47:56.229521 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:47:56.229521 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:47:56.229521 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:47:56.229521 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:47:56.229521 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:47:56.229521 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:47:56.229521 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:47:56.229521 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:47:56.229521 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 15:47:56.229521 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 15:47:56.229521 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 15:47:56.230434 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 15:47:56.230434 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:47:56.230434 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:47:56.230434 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:47:56.230434 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:47:56.230434 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:47:56.230434 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:47:56.230434 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:47:56.230434 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Jun 23 15:47:56.230434 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:47:56.230434 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:47:56.230434 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:47:56.230434 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:47:56.230434 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:47:56.230434 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:47:56.230434 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:47:56.230434 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:47:56.231367 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 15:47:56.231367 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 15:47:56.231367 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 15:47:56.231367 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Jun 23 15:47:56.231367 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 279/577] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Jun 23 15:47:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:47:56.231645 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697a9d26-4c02-49b7-bfcc-d43dc3e425df None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:56.256671 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60a5afd-caf8-4624-9f60-4ffff7defb09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:56.257395 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60a5afd-caf8-4624-9f60-4ffff7defb09 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:56.257678 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60a5afd-caf8-4624-9f60-4ffff7defb09 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:56.257953 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60a5afd-caf8-4624-9f60-4ffff7defb09 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:56.268077 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60a5afd-caf8-4624-9f60-4ffff7defb09 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 15:47:56.276073 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-d60a5afd-caf8-4624-9f60-4ffff7defb09 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 23 15:47:56.276073 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:47:56.276073 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:47:56.276073 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:47:56.276073 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:47:56.276073 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:47:56.276073 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:47:56.276073 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:47:56.276073 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:47:56.276073 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:47:56.276073 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:47:56.276073 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:47:56.276073 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 15:47:56.276073 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 15:47:56.276073 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 15:47:56.276073 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 15:47:56.277194 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 15:47:56.277194 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 15:47:56.277194 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 15:47:56.277194 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 15:47:56.277194 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:47:56.277194 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:47:56.277194 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:47:56.277194 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:47:56.277194 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:47:56.277194 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:47:56.277194 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:47:56.277194 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Jun 23 15:47:56.277194 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:47:56.277194 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:47:56.277194 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:47:56.277194 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:47:56.278329 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:47:56.278329 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:47:56.278329 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:47:56.278329 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:47:56.278329 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 15:47:56.278329 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 15:47:56.278329 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 15:47:56.278329 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 15:47:56.278329 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 15:47:56.278329 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 15:47:56.278329 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 23 15:47:56.278329 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Jun 23 15:47:56.278329 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 299/578] 10.222.0.46 () {56 vars in 1102 bytes} [Thu Jun 23 15:47:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:47:56.278329 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60a5afd-caf8-4624-9f60-4ffff7defb09 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:56.333020 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697a9d26-4c02-49b7-bfcc-d43dc3e425df None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:56.335444 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:56.368492 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6992949c-6038-49c0-acbc-f6ae39039465 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:56.369038 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:56.369151 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6992949c-6038-49c0-acbc-f6ae39039465 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:56.369436 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6992949c-6038-49c0-acbc-f6ae39039465 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:56.369716 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6992949c-6038-49c0-acbc-f6ae39039465 None admin] PATH_INFO: `/v3/projects` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:56.370828 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6992949c-6038-49c0-acbc-f6ae39039465 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:47:56.371287 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6992949c-6038-49c0-acbc-f6ae39039465 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:56.372712 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6992949c-6038-49c0-acbc-f6ae39039465 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:56.378677 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60a5afd-caf8-4624-9f60-4ffff7defb09 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:56.384319 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 280/579] 10.222.0.46 () {56 vars in 1148 bytes} [Thu Jun 23 15:47:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:47:56.385131 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6992949c-6038-49c0-acbc-f6ae39039465 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:56.414799 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6fc2d0-e26f-4786-bb2a-88e5c68171d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:56.415473 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6fc2d0-e26f-4786-bb2a-88e5c68171d1 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:56.415763 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6fc2d0-e26f-4786-bb2a-88e5c68171d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:56.416045 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6fc2d0-e26f-4786-bb2a-88e5c68171d1 None admin] PATH_INFO: `/v3/domains` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:56.416774 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6fc2d0-e26f-4786-bb2a-88e5c68171d1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:47:56.417085 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6fc2d0-e26f-4786-bb2a-88e5c68171d1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:56.418470 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6fc2d0-e26f-4786-bb2a-88e5c68171d1 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:56.430793 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 300/580] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Jun 23 15:47:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:47:56.431528 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6fc2d0-e26f-4786-bb2a-88e5c68171d1 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:56.489703 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6992949c-6038-49c0-acbc-f6ae39039465 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:56.524004 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a143ae-e159-4ade-824b-ff555ddc5b2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:56.524591 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a143ae-e159-4ade-824b-ff555ddc5b2d None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:56.524875 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a143ae-e159-4ade-824b-ff555ddc5b2d None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:56.525406 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a143ae-e159-4ade-824b-ff555ddc5b2d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:56.526240 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a143ae-e159-4ade-824b-ff555ddc5b2d None admin] RBAC: Adding query filter params (role.id=bfa8ecadc3d5483e821420aa0f0fb41e, scope.project.id=28ecdf5b474c44098c722f677d2a0b77, user.id=242bf02d14724fe4837fcce8730377f9) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:47:56.526554 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a143ae-e159-4ade-824b-ff555ddc5b2d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:56.528036 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a143ae-e159-4ade-824b-ff555ddc5b2d None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:56.528550 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6fc2d0-e26f-4786-bb2a-88e5c68171d1 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:56.533411 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 281/581] 10.222.0.46 () {56 vars in 1374 bytes} [Thu Jun 23 15:47:56 2022] GET /identity/v3/role_assignments?user.id=242bf02d14724fe4837fcce8730377f9&scope.project.id=28ecdf5b474c44098c722f677d2a0b77&role.id=bfa8ecadc3d5483e821420aa0f0fb41e => generated 259 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:47:56.534219 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a143ae-e159-4ade-824b-ff555ddc5b2d None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:56.567197 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:56.582003 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03efead8-34d6-4f14-82e3-43a45b342e94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:56.582723 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03efead8-34d6-4f14-82e3-43a45b342e94 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:56.583032 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03efead8-34d6-4f14-82e3-43a45b342e94 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:56.583311 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03efead8-34d6-4f14-82e3-43a45b342e94 None admin] PATH_INFO: `/v3/domains` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:56.584617 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03efead8-34d6-4f14-82e3-43a45b342e94 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:47:56.584930 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03efead8-34d6-4f14-82e3-43a45b342e94 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:56.586369 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03efead8-34d6-4f14-82e3-43a45b342e94 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:56.598895 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 301/582] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Jun 23 15:47:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:47:56.600327 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03efead8-34d6-4f14-82e3-43a45b342e94 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:56.614559 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:56.657667 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a143ae-e159-4ade-824b-ff555ddc5b2d None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:56.702721 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0351bf59-00ba-498f-87b2-73d9e2a9b07e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:56.702721 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0351bf59-00ba-498f-87b2-73d9e2a9b07e None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:56.702721 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0351bf59-00ba-498f-87b2-73d9e2a9b07e None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:56.702721 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0351bf59-00ba-498f-87b2-73d9e2a9b07e None admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:56.703193 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0351bf59-00ba-498f-87b2-73d9e2a9b07e None admin] RBAC: Adding query filter params (name=service) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:47:56.703193 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0351bf59-00ba-498f-87b2-73d9e2a9b07e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:56.703193 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0351bf59-00ba-498f-87b2-73d9e2a9b07e None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:56.712047 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 282/583] 10.222.0.46 () {56 vars in 1103 bytes} [Thu Jun 23 15:47:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:47:56.712862 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0351bf59-00ba-498f-87b2-73d9e2a9b07e None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:56.720616 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03efead8-34d6-4f14-82e3-43a45b342e94 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:56.756508 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20895a54-ab7a-40f9-a537-9c67dae089f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:56.757405 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20895a54-ab7a-40f9-a537-9c67dae089f4 None admin] REQUEST_METHOD: `PUT` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:56.757682 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20895a54-ab7a-40f9-a537-9c67dae089f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:56.757952 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20895a54-ab7a-40f9-a537-9c67dae089f4 None admin] PATH_INFO: `/v3/projects/28ecdf5b474c44098c722f677d2a0b77/users/705115d19bab424d894bea8b1b48a7f0/roles/bfa8ecadc3d5483e821420aa0f0fb41e` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:56.775480 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20895a54-ab7a-40f9-a537-9c67dae089f4 None admin] RBAC: Authorizing `identity:create_grant(project_id=28ecdf5b474c44098c722f677d2a0b77, user_id=705115d19bab424d894bea8b1b48a7f0, role_id=bfa8ecadc3d5483e821420aa0f0fb41e)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:56.777283 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20895a54-ab7a-40f9-a537-9c67dae089f4 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:56.791888 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 302/584] 10.222.0.46 () {58 vars in 1439 bytes} [Thu Jun 23 15:47:56 2022] PUT /identity/v3/projects/28ecdf5b474c44098c722f677d2a0b77/users/705115d19bab424d894bea8b1b48a7f0/roles/bfa8ecadc3d5483e821420aa0f0fb41e => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 15:47:56.793081 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20895a54-ab7a-40f9-a537-9c67dae089f4 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:56.833205 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:56.851737 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0351bf59-00ba-498f-87b2-73d9e2a9b07e None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:56.877426 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:56.903563 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccde8d5-2bad-4a0f-9a56-b598b262e546 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:56.904823 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccde8d5-2bad-4a0f-9a56-b598b262e546 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:56.905180 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccde8d5-2bad-4a0f-9a56-b598b262e546 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:56.905529 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccde8d5-2bad-4a0f-9a56-b598b262e546 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:56.916656 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccde8d5-2bad-4a0f-9a56-b598b262e546 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:56.918562 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccde8d5-2bad-4a0f-9a56-b598b262e546 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:56.929931 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20895a54-ab7a-40f9-a537-9c67dae089f4 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:56.937699 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-dccde8d5-2bad-4a0f-9a56-b598b262e546 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 23 15:47:56.937699 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:47:56.937699 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:47:56.937699 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:47:56.937699 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:47:56.937699 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:47:56.937699 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:47:56.937699 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:47:56.937699 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:47:56.937699 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:47:56.937699 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:47:56.937699 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:47:56.937699 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 15:47:56.937699 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 15:47:56.937699 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 15:47:56.937699 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 15:47:56.939129 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:47:56.939129 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:47:56.939129 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 15:47:56.939129 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 15:47:56.939129 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 15:47:56.939129 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 15:47:56.939129 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:47:56.939129 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:47:56.939129 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:47:56.939129 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:47:56.939129 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:47:56.939129 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Jun 23 15:47:56.939129 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:47:56.939129 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:47:56.939129 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:47:56.939129 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:47:56.939129 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:47:56.939129 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:47:56.940331 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:47:56.940331 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:47:56.940331 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 15:47:56.940331 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 15:47:56.940331 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 15:47:56.940331 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 15:47:56.940331 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 15:47:56.940331 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 15:47:56.940331 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 23 15:47:56.940331 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Jun 23 15:47:56.941136 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 283/585] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Jun 23 15:47:56 2022] GET /identity/v3/users/cinder => generated 84 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:47:56.941852 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccde8d5-2bad-4a0f-9a56-b598b262e546 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:56.976283 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7722474b-14e3-41f9-9f4f-533f7a951b5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:56.977190 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7722474b-14e3-41f9-9f4f-533f7a951b5b None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:56.977580 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7722474b-14e3-41f9-9f4f-533f7a951b5b None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:56.977944 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7722474b-14e3-41f9-9f4f-533f7a951b5b None admin] PATH_INFO: `/v3/users/glance` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:56.986327 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7722474b-14e3-41f9-9f4f-533f7a951b5b None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:56.987947 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7722474b-14e3-41f9-9f4f-533f7a951b5b None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:57.006611 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-7722474b-14e3-41f9-9f4f-533f7a951b5b None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 23 15:47:57.006611 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:47:57.006611 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:47:57.006611 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:47:57.006611 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:47:57.006611 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:47:57.006611 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:47:57.006611 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:47:57.006611 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:47:57.006611 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:47:57.006611 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:47:57.006611 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:47:57.006611 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 15:47:57.006611 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 15:47:57.006611 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 15:47:57.006611 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 15:47:57.007793 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:47:57.007793 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:47:57.007793 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 15:47:57.007793 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 15:47:57.007793 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 15:47:57.007793 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 15:47:57.007793 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:47:57.007793 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:47:57.007793 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:47:57.007793 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:47:57.007793 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:47:57.007793 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Jun 23 15:47:57.007793 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:47:57.007793 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:47:57.007793 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:47:57.007793 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:47:57.007793 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:47:57.007793 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:47:57.008921 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:47:57.008921 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:47:57.008921 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 15:47:57.008921 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 15:47:57.008921 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 15:47:57.008921 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 15:47:57.008921 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 15:47:57.008921 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 15:47:57.008921 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 23 15:47:57.008921 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Jun 23 15:47:57.010310 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 303/586] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Jun 23 15:47:56 2022] GET /identity/v3/users/glance => generated 84 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:47:57.011991 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7722474b-14e3-41f9-9f4f-533f7a951b5b None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:57.075682 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccde8d5-2bad-4a0f-9a56-b598b262e546 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:57.120348 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7ac5fb-c206-48f6-9557-82461dab964e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:57.121780 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7ac5fb-c206-48f6-9557-82461dab964e None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:57.122139 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7ac5fb-c206-48f6-9557-82461dab964e None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:57.122489 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7ac5fb-c206-48f6-9557-82461dab964e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:57.123642 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7722474b-14e3-41f9-9f4f-533f7a951b5b None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:57.136575 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7ac5fb-c206-48f6-9557-82461dab964e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:57.138275 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7ac5fb-c206-48f6-9557-82461dab964e None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:57.145486 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:57.161078 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-bc7ac5fb-c206-48f6-9557-82461dab964e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 15:47:57.161078 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:47:57.161078 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:47:57.161078 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:47:57.161078 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:47:57.161078 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:47:57.161078 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:47:57.161078 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:47:57.161078 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:47:57.161078 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:47:57.161078 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:47:57.161078 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:47:57.161078 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 15:47:57.161078 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 15:47:57.161078 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 15:47:57.162211 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 15:47:57.162211 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:47:57.162211 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:47:57.162211 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:47:57.162211 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:47:57.162211 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:47:57.162211 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:47:57.162211 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:47:57.162211 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Jun 23 15:47:57.162211 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:47:57.162211 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:47:57.162211 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:47:57.162211 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:47:57.162211 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:47:57.162211 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:47:57.162211 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:47:57.162211 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:47:57.163138 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 15:47:57.163138 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 15:47:57.163138 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 15:47:57.163138 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Jun 23 15:47:57.163550 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 284/587] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Jun 23 15:47:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:47:57.164529 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7ac5fb-c206-48f6-9557-82461dab964e None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:57.165955 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35356d33-6c17-449a-aa60-ad7b54e23f7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:57.166906 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35356d33-6c17-449a-aa60-ad7b54e23f7a None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:57.167371 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35356d33-6c17-449a-aa60-ad7b54e23f7a None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:57.167474 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:57.168007 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35356d33-6c17-449a-aa60-ad7b54e23f7a None admin] PATH_INFO: `/v3/users` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:57.169073 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35356d33-6c17-449a-aa60-ad7b54e23f7a None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:47:57.169646 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35356d33-6c17-449a-aa60-ad7b54e23f7a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:57.171548 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35356d33-6c17-449a-aa60-ad7b54e23f7a None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:57.194836 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 304/588] 10.222.0.46 () {56 vars in 1137 bytes} [Thu Jun 23 15:47:57 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:47:57.196340 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35356d33-6c17-449a-aa60-ad7b54e23f7a None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:57.287654 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7ac5fb-c206-48f6-9557-82461dab964e None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:57.313863 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35356d33-6c17-449a-aa60-ad7b54e23f7a None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:57.324624 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c65251-1ff6-4575-b2c5-21540d4f1423 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:57.325309 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c65251-1ff6-4575-b2c5-21540d4f1423 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:57.325597 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c65251-1ff6-4575-b2c5-21540d4f1423 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:57.325933 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c65251-1ff6-4575-b2c5-21540d4f1423 None admin] PATH_INFO: `/v3/users` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:57.326842 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c65251-1ff6-4575-b2c5-21540d4f1423 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:47:57.327163 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c65251-1ff6-4575-b2c5-21540d4f1423 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:57.328610 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c65251-1ff6-4575-b2c5-21540d4f1423 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:57.347590 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 285/589] 10.222.0.46 () {56 vars in 1137 bytes} [Thu Jun 23 15:47:57 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:47:57.348201 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c65251-1ff6-4575-b2c5-21540d4f1423 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:57.361938 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e34b45b-459d-4e84-bbb6-556c0c37c5be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:57.363206 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e34b45b-459d-4e84-bbb6-556c0c37c5be None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:57.363493 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e34b45b-459d-4e84-bbb6-556c0c37c5be None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:57.363773 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e34b45b-459d-4e84-bbb6-556c0c37c5be None admin] PATH_INFO: `/v3/domains` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:57.364529 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e34b45b-459d-4e84-bbb6-556c0c37c5be None admin] RBAC: Adding query filter params (name=Default) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:47:57.364815 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e34b45b-459d-4e84-bbb6-556c0c37c5be None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:57.366773 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e34b45b-459d-4e84-bbb6-556c0c37c5be None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:57.380719 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 305/590] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Jun 23 15:47:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:47:57.382286 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e34b45b-459d-4e84-bbb6-556c0c37c5be None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:57.417676 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:57.432853 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:57.466612 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c65251-1ff6-4575-b2c5-21540d4f1423 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:57.512156 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a5c7b3-ac55-4e7f-b75b-f9a27d7aa321 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:57.513238 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a5c7b3-ac55-4e7f-b75b-f9a27d7aa321 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:57.513524 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a5c7b3-ac55-4e7f-b75b-f9a27d7aa321 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:57.513964 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a5c7b3-ac55-4e7f-b75b-f9a27d7aa321 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:57.524586 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e34b45b-459d-4e84-bbb6-556c0c37c5be None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:57.527278 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a5c7b3-ac55-4e7f-b75b-f9a27d7aa321 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:57.529532 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a5c7b3-ac55-4e7f-b75b-f9a27d7aa321 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:57.551674 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-77a5c7b3-ac55-4e7f-b75b-f9a27d7aa321 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 15:47:57.551674 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:47:57.551674 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:47:57.551674 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:47:57.551674 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:47:57.551674 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:47:57.551674 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:47:57.551674 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:47:57.551674 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:47:57.551674 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:47:57.551674 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:47:57.551674 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:47:57.551674 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 15:47:57.551674 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 15:47:57.551674 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 15:47:57.552625 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 15:47:57.552625 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:47:57.552625 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:47:57.552625 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:47:57.552625 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:47:57.552625 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:47:57.552625 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:47:57.552625 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:47:57.552625 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Jun 23 15:47:57.552625 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:47:57.552625 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:47:57.552625 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:47:57.552625 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:47:57.552625 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:47:57.552625 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:47:57.552625 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:47:57.552625 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:47:57.554044 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 15:47:57.554044 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 15:47:57.554044 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 15:47:57.554044 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Jun 23 15:47:57.554044 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 286/591] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Jun 23 15:47:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:47:57.554885 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a5c7b3-ac55-4e7f-b75b-f9a27d7aa321 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:57.569767 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94909a0-78f6-4a91-89f2-d7f603c62723 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:57.571044 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94909a0-78f6-4a91-89f2-d7f603c62723 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:57.571617 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94909a0-78f6-4a91-89f2-d7f603c62723 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:57.574964 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94909a0-78f6-4a91-89f2-d7f603c62723 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:57.589819 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94909a0-78f6-4a91-89f2-d7f603c62723 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:57.592251 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94909a0-78f6-4a91-89f2-d7f603c62723 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:57.619422 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-b94909a0-78f6-4a91-89f2-d7f603c62723 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 15:47:57.619422 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:47:57.619422 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:47:57.619422 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:47:57.619422 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:47:57.619422 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:47:57.619422 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:47:57.619422 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:47:57.619422 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:47:57.619422 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:47:57.619422 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:47:57.619422 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:47:57.619422 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 15:47:57.619422 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 15:47:57.619422 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 15:47:57.620356 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 15:47:57.620356 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:47:57.620356 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:47:57.620356 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:47:57.620356 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:47:57.620356 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:47:57.620356 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:47:57.620356 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:47:57.620356 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Jun 23 15:47:57.620356 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:47:57.620356 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:47:57.620356 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:47:57.620356 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:47:57.620356 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:47:57.620356 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:47:57.620356 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:47:57.620356 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:47:57.621310 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 15:47:57.621310 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 15:47:57.621310 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 15:47:57.621310 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Jun 23 15:47:57.622850 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 306/592] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Jun 23 15:47:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:47:57.623697 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94909a0-78f6-4a91-89f2-d7f603c62723 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:57.682933 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a5c7b3-ac55-4e7f-b75b-f9a27d7aa321 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:57.698912 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:57.723424 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b33dd6d-ac79-4bfe-a8a5-d52a8581d68c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:57.725119 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:57.725809 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b33dd6d-ac79-4bfe-a8a5-d52a8581d68c None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:57.726302 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b33dd6d-ac79-4bfe-a8a5-d52a8581d68c None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:57.726786 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b33dd6d-ac79-4bfe-a8a5-d52a8581d68c None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:57.735396 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94909a0-78f6-4a91-89f2-d7f603c62723 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:57.735798 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b33dd6d-ac79-4bfe-a8a5-d52a8581d68c None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:57.737514 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b33dd6d-ac79-4bfe-a8a5-d52a8581d68c None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:57.755644 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-6b33dd6d-ac79-4bfe-a8a5-d52a8581d68c None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 23 15:47:57.755644 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:47:57.755644 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:47:57.755644 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:47:57.755644 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:47:57.755644 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:47:57.755644 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:47:57.755644 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:47:57.755644 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:47:57.755644 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:47:57.755644 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:47:57.755644 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:47:57.755644 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 15:47:57.755644 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 15:47:57.755644 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 15:47:57.755644 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 15:47:57.756837 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:47:57.756837 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:47:57.756837 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 15:47:57.756837 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 15:47:57.756837 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 15:47:57.756837 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 15:47:57.756837 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:47:57.756837 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:47:57.756837 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:47:57.756837 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:47:57.756837 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:47:57.756837 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Jun 23 15:47:57.756837 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:47:57.756837 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:47:57.756837 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:47:57.756837 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:47:57.756837 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:47:57.756837 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:47:57.757913 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:47:57.757913 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:47:57.757913 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 15:47:57.757913 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 15:47:57.757913 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 15:47:57.757913 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 15:47:57.757913 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 15:47:57.757913 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 15:47:57.757913 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 23 15:47:57.757913 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Jun 23 15:47:57.760112 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 287/593] 10.222.0.46 () {56 vars in 1102 bytes} [Thu Jun 23 15:47:57 2022] GET /identity/v3/users/neutron => generated 85 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:47:57.761130 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b33dd6d-ac79-4bfe-a8a5-d52a8581d68c None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:57.770833 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0ed385-a551-43c0-8b68-5a028483f116 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:57.771593 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0ed385-a551-43c0-8b68-5a028483f116 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:57.771879 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0ed385-a551-43c0-8b68-5a028483f116 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:57.772157 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0ed385-a551-43c0-8b68-5a028483f116 None admin] PATH_INFO: `/v3/domains` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:57.772942 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0ed385-a551-43c0-8b68-5a028483f116 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:47:57.773270 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0ed385-a551-43c0-8b68-5a028483f116 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:57.774766 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0ed385-a551-43c0-8b68-5a028483f116 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:57.788250 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 307/594] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Jun 23 15:47:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:47:57.789096 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0ed385-a551-43c0-8b68-5a028483f116 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:57.878350 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b33dd6d-ac79-4bfe-a8a5-d52a8581d68c None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:57.895990 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0ed385-a551-43c0-8b68-5a028483f116 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:57.918151 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91a0079-2d41-42e6-8eea-fe67c3ca6963 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:57.919182 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91a0079-2d41-42e6-8eea-fe67c3ca6963 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:57.919659 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91a0079-2d41-42e6-8eea-fe67c3ca6963 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:57.920077 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91a0079-2d41-42e6-8eea-fe67c3ca6963 None admin] PATH_INFO: `/v3/domains` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:57.920947 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91a0079-2d41-42e6-8eea-fe67c3ca6963 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:47:57.921399 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91a0079-2d41-42e6-8eea-fe67c3ca6963 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:57.923074 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91a0079-2d41-42e6-8eea-fe67c3ca6963 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:57.940601 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3682589e-91de-4600-82c1-c3d8d43be274 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:57.940601 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3682589e-91de-4600-82c1-c3d8d43be274 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:57.940601 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3682589e-91de-4600-82c1-c3d8d43be274 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:57.940601 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3682589e-91de-4600-82c1-c3d8d43be274 None admin] PATH_INFO: `/v3/users` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:57.941011 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3682589e-91de-4600-82c1-c3d8d43be274 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:47:57.941011 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3682589e-91de-4600-82c1-c3d8d43be274 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:57.941011 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3682589e-91de-4600-82c1-c3d8d43be274 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:57.941265 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 288/595] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Jun 23 15:47:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:47:57.942267 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91a0079-2d41-42e6-8eea-fe67c3ca6963 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:57.959235 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 308/596] 10.222.0.46 () {56 vars in 1139 bytes} [Thu Jun 23 15:47:57 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:47:57.960077 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3682589e-91de-4600-82c1-c3d8d43be274 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:57.970563 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:57.980316 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:58.063378 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91a0079-2d41-42e6-8eea-fe67c3ca6963 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:58.067278 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3682589e-91de-4600-82c1-c3d8d43be274 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:58.101855 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748d8603-8a88-4cd0-929b-a032dd6de82e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:58.102943 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748d8603-8a88-4cd0-929b-a032dd6de82e None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:58.103374 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748d8603-8a88-4cd0-929b-a032dd6de82e None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:58.103488 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be9d892-78d1-4935-aecd-e42bc57162a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:58.103929 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748d8603-8a88-4cd0-929b-a032dd6de82e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:58.104195 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be9d892-78d1-4935-aecd-e42bc57162a2 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:58.104479 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be9d892-78d1-4935-aecd-e42bc57162a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:58.104758 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be9d892-78d1-4935-aecd-e42bc57162a2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:58.111762 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be9d892-78d1-4935-aecd-e42bc57162a2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:58.112006 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748d8603-8a88-4cd0-929b-a032dd6de82e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:58.113211 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be9d892-78d1-4935-aecd-e42bc57162a2 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:58.113793 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748d8603-8a88-4cd0-929b-a032dd6de82e None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:58.127526 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-1be9d892-78d1-4935-aecd-e42bc57162a2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 15:47:58.127526 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:47:58.127526 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:47:58.127526 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:47:58.127526 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:47:58.127526 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:47:58.127526 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:47:58.127526 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:47:58.127526 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:47:58.127526 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:47:58.127526 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:47:58.127526 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:47:58.127526 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 15:47:58.127526 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 15:47:58.127526 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 15:47:58.128663 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 15:47:58.128663 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:47:58.128663 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:47:58.128663 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:47:58.128663 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:47:58.128663 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:47:58.128663 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:47:58.128663 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:47:58.128663 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Jun 23 15:47:58.128663 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:47:58.128663 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:47:58.128663 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:47:58.128663 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:47:58.128663 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:47:58.128663 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:47:58.128663 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:47:58.128663 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:47:58.129755 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 15:47:58.129755 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 15:47:58.129755 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 15:47:58.129755 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 15:47:58.129755 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 15:47:58.129755 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 15:47:58.129755 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 15:47:58.129755 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Jun 23 15:47:58.130203 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-748d8603-8a88-4cd0-929b-a032dd6de82e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 15:47:58.130203 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:47:58.130203 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:47:58.130203 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:47:58.130203 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:47:58.130203 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:47:58.130203 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:47:58.130203 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:47:58.130203 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:47:58.130203 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:47:58.130203 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:47:58.130203 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:47:58.130203 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 15:47:58.130203 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 15:47:58.130203 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 15:47:58.131206 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 15:47:58.131206 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:47:58.131206 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:47:58.131206 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:47:58.131206 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:47:58.131206 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:47:58.131206 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:47:58.131206 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:47:58.131206 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Jun 23 15:47:58.131206 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:47:58.131206 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:47:58.131206 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:47:58.131206 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:47:58.131206 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:47:58.131206 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:47:58.131206 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:47:58.131206 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:47:58.132097 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 15:47:58.132097 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 15:47:58.132097 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 15:47:58.132097 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 15:47:58.132097 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 15:47:58.132097 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 15:47:58.132097 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 15:47:58.132097 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Jun 23 15:47:58.132097 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 289/597] 10.222.0.46 () {56 vars in 1111 bytes} [Thu Jun 23 15:47:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:47:58.132097 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748d8603-8a88-4cd0-929b-a032dd6de82e None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:58.133938 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 309/598] 10.222.0.46 () {56 vars in 1111 bytes} [Thu Jun 23 15:47:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:47:58.142903 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be9d892-78d1-4935-aecd-e42bc57162a2 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:58.215565 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:58.238267 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:58.248174 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748d8603-8a88-4cd0-929b-a032dd6de82e None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:58.256947 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be9d892-78d1-4935-aecd-e42bc57162a2 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:58.285831 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254812ca-70da-4378-819d-5d7577201ceb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:58.286779 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254812ca-70da-4378-819d-5d7577201ceb None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:58.287122 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254812ca-70da-4378-819d-5d7577201ceb None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:58.287930 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254812ca-70da-4378-819d-5d7577201ceb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:58.296703 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b030c14-c1c6-4277-bb1e-7ddea2efd749 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:58.297580 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b030c14-c1c6-4277-bb1e-7ddea2efd749 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:58.298031 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b030c14-c1c6-4277-bb1e-7ddea2efd749 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:58.298462 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b030c14-c1c6-4277-bb1e-7ddea2efd749 None admin] PATH_INFO: `/v3/projects` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:58.298652 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254812ca-70da-4378-819d-5d7577201ceb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:58.299573 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b030c14-c1c6-4277-bb1e-7ddea2efd749 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:47:58.300039 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b030c14-c1c6-4277-bb1e-7ddea2efd749 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:58.300170 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254812ca-70da-4378-819d-5d7577201ceb None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:58.302048 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b030c14-c1c6-4277-bb1e-7ddea2efd749 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:58.320021 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-254812ca-70da-4378-819d-5d7577201ceb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 15:47:58.320021 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:47:58.320021 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:47:58.320021 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:47:58.320021 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:47:58.320021 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:47:58.320021 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:47:58.320021 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:47:58.320021 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:47:58.320021 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:47:58.320021 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:47:58.320021 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:47:58.320021 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 15:47:58.320021 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 15:47:58.320021 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 15:47:58.321140 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 15:47:58.321140 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:47:58.321140 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:47:58.321140 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:47:58.321140 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:47:58.321140 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:47:58.321140 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:47:58.321140 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:47:58.321140 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Jun 23 15:47:58.321140 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:47:58.321140 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:47:58.321140 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:47:58.321140 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:47:58.321140 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:47:58.321140 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:47:58.321140 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:47:58.321140 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:47:58.322089 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 15:47:58.322089 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 15:47:58.322089 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 15:47:58.322089 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Jun 23 15:47:58.322622 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 310/599] 10.222.0.46 () {56 vars in 1148 bytes} [Thu Jun 23 15:47:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 15:47:58.324000 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b030c14-c1c6-4277-bb1e-7ddea2efd749 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:58.324886 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 290/600] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Jun 23 15:47:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:47:58.325716 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254812ca-70da-4378-819d-5d7577201ceb None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:58.434832 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254812ca-70da-4378-819d-5d7577201ceb None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:58.435981 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b030c14-c1c6-4277-bb1e-7ddea2efd749 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:58.467463 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:58.476041 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492067ea-9d42-4b6e-a23d-4123012a2bc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:58.476920 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492067ea-9d42-4b6e-a23d-4123012a2bc4 None admin] REQUEST_METHOD: `PUT` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:58.477231 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492067ea-9d42-4b6e-a23d-4123012a2bc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:58.477476 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492067ea-9d42-4b6e-a23d-4123012a2bc4 None admin] PATH_INFO: `/v3/projects/28ecdf5b474c44098c722f677d2a0b77/users/ce18eaaba3a6477684fff1fd98ec8b51/roles/bfa8ecadc3d5483e821420aa0f0fb41e` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:58.478946 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9c7558-341f-4999-b3eb-3909421e1682 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:58.479933 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9c7558-341f-4999-b3eb-3909421e1682 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:58.480398 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9c7558-341f-4999-b3eb-3909421e1682 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:58.480916 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9c7558-341f-4999-b3eb-3909421e1682 None admin] PATH_INFO: `/v3/projects` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:58.482005 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9c7558-341f-4999-b3eb-3909421e1682 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:47:58.482616 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9c7558-341f-4999-b3eb-3909421e1682 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:58.484247 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9c7558-341f-4999-b3eb-3909421e1682 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:58.493432 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492067ea-9d42-4b6e-a23d-4123012a2bc4 None admin] RBAC: Authorizing `identity:create_grant(project_id=28ecdf5b474c44098c722f677d2a0b77, user_id=ce18eaaba3a6477684fff1fd98ec8b51, role_id=bfa8ecadc3d5483e821420aa0f0fb41e)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:58.495029 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492067ea-9d42-4b6e-a23d-4123012a2bc4 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:58.497274 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:58.499647 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 311/601] 10.222.0.46 () {56 vars in 1148 bytes} [Thu Jun 23 15:47:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:47:58.500745 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9c7558-341f-4999-b3eb-3909421e1682 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:58.522426 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 291/602] 10.222.0.46 () {58 vars in 1439 bytes} [Thu Jun 23 15:47:58 2022] PUT /identity/v3/projects/28ecdf5b474c44098c722f677d2a0b77/users/ce18eaaba3a6477684fff1fd98ec8b51/roles/bfa8ecadc3d5483e821420aa0f0fb41e => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 15:47:58.522426 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492067ea-9d42-4b6e-a23d-4123012a2bc4 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:58.647699 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492067ea-9d42-4b6e-a23d-4123012a2bc4 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:58.687719 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230d39b6-a0cc-4bd1-8f56-40c3bcd25f90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:58.688723 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230d39b6-a0cc-4bd1-8f56-40c3bcd25f90 None admin] REQUEST_METHOD: `PUT` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:58.689112 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230d39b6-a0cc-4bd1-8f56-40c3bcd25f90 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:58.689588 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230d39b6-a0cc-4bd1-8f56-40c3bcd25f90 None admin] PATH_INFO: `/v3/projects/28ecdf5b474c44098c722f677d2a0b77/users/83970271286e451e9008bbf07cd3fa86/roles/bfa8ecadc3d5483e821420aa0f0fb41e` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:58.694369 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9c7558-341f-4999-b3eb-3909421e1682 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:58.707885 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230d39b6-a0cc-4bd1-8f56-40c3bcd25f90 None admin] RBAC: Authorizing `identity:create_grant(project_id=28ecdf5b474c44098c722f677d2a0b77, user_id=83970271286e451e9008bbf07cd3fa86, role_id=bfa8ecadc3d5483e821420aa0f0fb41e)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:58.710250 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230d39b6-a0cc-4bd1-8f56-40c3bcd25f90 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:58.729295 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 292/603] 10.222.0.46 () {58 vars in 1439 bytes} [Thu Jun 23 15:47:58 2022] PUT /identity/v3/projects/28ecdf5b474c44098c722f677d2a0b77/users/83970271286e451e9008bbf07cd3fa86/roles/bfa8ecadc3d5483e821420aa0f0fb41e => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 15:47:58.736908 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb5332f-be33-475e-a9e5-8bacc0f53b35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:58.737624 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb5332f-be33-475e-a9e5-8bacc0f53b35 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:58.737914 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb5332f-be33-475e-a9e5-8bacc0f53b35 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:58.738199 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb5332f-be33-475e-a9e5-8bacc0f53b35 None admin] PATH_INFO: `/v3/domains` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:58.739099 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb5332f-be33-475e-a9e5-8bacc0f53b35 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:47:58.739629 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb5332f-be33-475e-a9e5-8bacc0f53b35 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:58.739750 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2143110c-85cf-426c-903b-ed8c65269a12 None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:58.740631 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2143110c-85cf-426c-903b-ed8c65269a12 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:58.741003 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2143110c-85cf-426c-903b-ed8c65269a12 None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:58.741995 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb5332f-be33-475e-a9e5-8bacc0f53b35 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:58.742876 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 293/604] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jun 23 15:47:58 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 15:47:58.748513 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eba3727-3df9-4b93-9a48-ba4eafda5f8b None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:58.748806 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eba3727-3df9-4b93-9a48-ba4eafda5f8b None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:58.749244 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eba3727-3df9-4b93-9a48-ba4eafda5f8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:58.758802 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 312/605] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Jun 23 15:47:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:47:58.773733 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb5332f-be33-475e-a9e5-8bacc0f53b35 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:58.778763 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-0eba3727-3df9-4b93-9a48-ba4eafda5f8b None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:47:58.780040 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-0eba3727-3df9-4b93-9a48-ba4eafda5f8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:47:58.838262 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:58.846817 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:58.876481 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 294/606] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jun 23 15:47:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 15:47:58.910543 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eba3727-3df9-4b93-9a48-ba4eafda5f8b None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:58.916695 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb5332f-be33-475e-a9e5-8bacc0f53b35 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:58.960795 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c9cd63-7c51-4897-a95c-ed759d62e17f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:58.962064 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c9cd63-7c51-4897-a95c-ed759d62e17f None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:58.963390 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c9cd63-7c51-4897-a95c-ed759d62e17f None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:58.964539 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c9cd63-7c51-4897-a95c-ed759d62e17f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:58.975968 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c9cd63-7c51-4897-a95c-ed759d62e17f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:58.977791 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c9cd63-7c51-4897-a95c-ed759d62e17f None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:59.000088 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-a9c9cd63-7c51-4897-a95c-ed759d62e17f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 15:47:59.000088 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:47:59.000088 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:47:59.000088 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:47:59.000088 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:47:59.000088 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:47:59.000088 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:47:59.000088 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:47:59.000088 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:47:59.000088 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:47:59.000088 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:47:59.000088 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:47:59.000088 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 15:47:59.000088 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 15:47:59.000088 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 15:47:59.001083 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 15:47:59.001083 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:47:59.001083 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:47:59.001083 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:47:59.001083 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:47:59.001083 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:47:59.001083 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:47:59.001083 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:47:59.001083 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Jun 23 15:47:59.001083 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:47:59.001083 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:47:59.001083 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:47:59.001083 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:47:59.001083 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:47:59.001083 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:47:59.001083 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:47:59.001083 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:47:59.002000 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 15:47:59.002000 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 15:47:59.002000 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 15:47:59.002000 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 15:47:59.002000 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 15:47:59.002000 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 15:47:59.002000 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 15:47:59.002000 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Jun 23 15:47:59.002000 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 313/607] 10.222.0.46 () {56 vars in 1111 bytes} [Thu Jun 23 15:47:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:47:59.008399 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c9cd63-7c51-4897-a95c-ed759d62e17f None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:59.067920 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eba3727-3df9-4b93-9a48-ba4eafda5f8b None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:59.109966 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4745d834-c3d7-4e8a-8ab5-14e9282c804d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:59.111398 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4745d834-c3d7-4e8a-8ab5-14e9282c804d None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:59.112136 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4745d834-c3d7-4e8a-8ab5-14e9282c804d None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:59.112610 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4745d834-c3d7-4e8a-8ab5-14e9282c804d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:59.128299 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4745d834-c3d7-4e8a-8ab5-14e9282c804d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 15:47:59.129666 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c9cd63-7c51-4897-a95c-ed759d62e17f None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:59.138914 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-4745d834-c3d7-4e8a-8ab5-14e9282c804d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 23 15:47:59.138914 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:47:59.138914 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:47:59.138914 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:47:59.138914 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:47:59.138914 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:47:59.138914 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:47:59.138914 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:47:59.138914 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:47:59.138914 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:47:59.138914 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:47:59.138914 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:47:59.138914 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 15:47:59.138914 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 15:47:59.138914 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 15:47:59.138914 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 15:47:59.139868 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 15:47:59.139868 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 15:47:59.139868 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 15:47:59.139868 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 15:47:59.139868 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:47:59.139868 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:47:59.139868 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:47:59.139868 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:47:59.139868 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:47:59.139868 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:47:59.139868 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:47:59.139868 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Jun 23 15:47:59.139868 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:47:59.139868 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:47:59.139868 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:47:59.139868 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:47:59.140698 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:47:59.140698 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:47:59.140698 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:47:59.140698 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:47:59.140698 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 15:47:59.140698 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 15:47:59.140698 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 15:47:59.140698 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 15:47:59.140698 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 15:47:59.140698 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 15:47:59.140698 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 23 15:47:59.140698 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Jun 23 15:47:59.142330 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:59.145577 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 295/608] 10.222.0.46 () {56 vars in 1102 bytes} [Thu Jun 23 15:47:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:47:59.147946 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7ceedc-e100-4d6b-9a62-4459124f4ec8 None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:59.148403 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7ceedc-e100-4d6b-9a62-4459124f4ec8 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:59.148831 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7ceedc-e100-4d6b-9a62-4459124f4ec8 None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:59.150937 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 296/609] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jun 23 15:47:59 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 15:47:59.151968 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7ceedc-e100-4d6b-9a62-4459124f4ec8 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:59.185643 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a1539c-6c91-4db2-bb91-095ee49c6636 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:59.187049 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a1539c-6c91-4db2-bb91-095ee49c6636 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:59.187744 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a1539c-6c91-4db2-bb91-095ee49c6636 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:59.188182 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a1539c-6c91-4db2-bb91-095ee49c6636 None admin] PATH_INFO: `/v3/projects` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:59.189497 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a1539c-6c91-4db2-bb91-095ee49c6636 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:47:59.190381 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a1539c-6c91-4db2-bb91-095ee49c6636 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:59.192497 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a1539c-6c91-4db2-bb91-095ee49c6636 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:59.194770 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:59.212847 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 314/610] 10.222.0.46 () {56 vars in 1148 bytes} [Thu Jun 23 15:47:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 15:47:59.215156 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847cadc6-fde4-41e3-9047-d06ab20847e1 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:59.215692 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847cadc6-fde4-41e3-9047-d06ab20847e1 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:59.216180 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847cadc6-fde4-41e3-9047-d06ab20847e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:59.237270 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-847cadc6-fde4-41e3-9047-d06ab20847e1 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:47:59.238471 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-847cadc6-fde4-41e3-9047-d06ab20847e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:47:59.290427 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7ceedc-e100-4d6b-9a62-4459124f4ec8 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:59.320245 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 315/611] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jun 23 15:47:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 15:47:59.321967 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847cadc6-fde4-41e3-9047-d06ab20847e1 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:59.335591 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bbb992-6ad2-489f-857b-081ef836329d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:59.339923 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bbb992-6ad2-489f-857b-081ef836329d None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:59.340212 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bbb992-6ad2-489f-857b-081ef836329d None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:59.340494 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bbb992-6ad2-489f-857b-081ef836329d None admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:59.341266 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1bbb992-6ad2-489f-857b-081ef836329d None admin] RBAC: Adding query filter params (name=service) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:47:59.341576 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1bbb992-6ad2-489f-857b-081ef836329d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:59.343988 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1bbb992-6ad2-489f-857b-081ef836329d None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:59.355198 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 297/612] 10.222.0.46 () {56 vars in 1103 bytes} [Thu Jun 23 15:47:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:47:59.356518 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bbb992-6ad2-489f-857b-081ef836329d None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:59.457385 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847cadc6-fde4-41e3-9047-d06ab20847e1 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:59.472143 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:59.486361 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bbb992-6ad2-489f-857b-081ef836329d None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:59.506084 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef29ff87-9427-4bd3-ac79-4df04b74ece8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:59.507786 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef29ff87-9427-4bd3-ac79-4df04b74ece8 None admin] REQUEST_METHOD: `PUT` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:59.508282 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef29ff87-9427-4bd3-ac79-4df04b74ece8 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:59.508750 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef29ff87-9427-4bd3-ac79-4df04b74ece8 None admin] PATH_INFO: `/v3/projects/28ecdf5b474c44098c722f677d2a0b77/users/51784064951a4619ae09b9167fb0fd0d/roles/bfa8ecadc3d5483e821420aa0f0fb41e` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:59.511049 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:59.530781 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef29ff87-9427-4bd3-ac79-4df04b74ece8 None admin] RBAC: Authorizing `identity:create_grant(project_id=28ecdf5b474c44098c722f677d2a0b77, user_id=51784064951a4619ae09b9167fb0fd0d, role_id=bfa8ecadc3d5483e821420aa0f0fb41e)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:59.532258 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef29ff87-9427-4bd3-ac79-4df04b74ece8 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:59.533805 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dff58f1-2f2d-4fcd-93f1-4e8c1410cb87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:59.536028 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dff58f1-2f2d-4fcd-93f1-4e8c1410cb87 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:59.536688 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dff58f1-2f2d-4fcd-93f1-4e8c1410cb87 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:59.537830 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dff58f1-2f2d-4fcd-93f1-4e8c1410cb87 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:59.547215 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 316/613] 10.222.0.46 () {58 vars in 1439 bytes} [Thu Jun 23 15:47:59 2022] PUT /identity/v3/projects/28ecdf5b474c44098c722f677d2a0b77/users/51784064951a4619ae09b9167fb0fd0d/roles/bfa8ecadc3d5483e821420aa0f0fb41e => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 15:47:59.548135 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef29ff87-9427-4bd3-ac79-4df04b74ece8 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:59.551072 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dff58f1-2f2d-4fcd-93f1-4e8c1410cb87 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 15:47:59.560163 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-6dff58f1-2f2d-4fcd-93f1-4e8c1410cb87 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 23 15:47:59.560163 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:47:59.560163 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:47:59.560163 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:47:59.560163 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:47:59.560163 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:47:59.560163 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:47:59.560163 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:47:59.560163 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:47:59.560163 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:47:59.560163 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:47:59.560163 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:47:59.560163 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 15:47:59.560163 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 15:47:59.560163 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 15:47:59.560163 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 15:47:59.561169 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 15:47:59.561169 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 15:47:59.561169 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 15:47:59.561169 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 15:47:59.561169 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:47:59.561169 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:47:59.561169 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:47:59.561169 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:47:59.561169 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:47:59.561169 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:47:59.561169 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:47:59.561169 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Jun 23 15:47:59.561169 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:47:59.561169 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:47:59.561169 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:47:59.561169 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:47:59.562026 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:47:59.562026 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:47:59.562026 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:47:59.562026 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:47:59.562026 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 15:47:59.562026 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 15:47:59.562026 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 15:47:59.562026 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 15:47:59.562026 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 15:47:59.562026 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 15:47:59.562026 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 23 15:47:59.562026 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Jun 23 15:47:59.569791 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 298/614] 10.222.0.46 () {56 vars in 1102 bytes} [Thu Jun 23 15:47:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:47:59.572038 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dff58f1-2f2d-4fcd-93f1-4e8c1410cb87 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:59.680214 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef29ff87-9427-4bd3-ac79-4df04b74ece8 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:59.696474 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dff58f1-2f2d-4fcd-93f1-4e8c1410cb87 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:59.739784 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948018d4-70f8-48db-80d6-b8c1a9c471df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:59.740910 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948018d4-70f8-48db-80d6-b8c1a9c471df None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:59.741403 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948018d4-70f8-48db-80d6-b8c1a9c471df None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:59.741885 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948018d4-70f8-48db-80d6-b8c1a9c471df None admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:59.743006 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948018d4-70f8-48db-80d6-b8c1a9c471df None admin] RBAC: Adding query filter params (name=service) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:47:59.743665 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948018d4-70f8-48db-80d6-b8c1a9c471df None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:59.745681 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948018d4-70f8-48db-80d6-b8c1a9c471df None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:59.759609 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97c1ff7-8256-4f06-a439-51372930ee0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:59.760458 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97c1ff7-8256-4f06-a439-51372930ee0d None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:59.760746 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97c1ff7-8256-4f06-a439-51372930ee0d None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:59.761026 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97c1ff7-8256-4f06-a439-51372930ee0d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:59.762769 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 299/615] 10.222.0.46 () {56 vars in 1103 bytes} [Thu Jun 23 15:47:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 15:47:59.765550 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948018d4-70f8-48db-80d6-b8c1a9c471df None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:59.779098 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97c1ff7-8256-4f06-a439-51372930ee0d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:59.780575 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97c1ff7-8256-4f06-a439-51372930ee0d None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:59.793992 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:59.794549 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:47:59.814556 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-a97c1ff7-8256-4f06-a439-51372930ee0d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 15:47:59.814556 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:47:59.814556 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:47:59.814556 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:47:59.814556 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:47:59.814556 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:47:59.814556 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:47:59.814556 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:47:59.814556 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:47:59.814556 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:47:59.814556 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:47:59.814556 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:47:59.814556 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 15:47:59.814556 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 15:47:59.814556 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 15:47:59.815741 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 15:47:59.815741 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:47:59.815741 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:47:59.815741 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:47:59.815741 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:47:59.815741 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:47:59.815741 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:47:59.815741 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:47:59.815741 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Jun 23 15:47:59.815741 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:47:59.815741 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:47:59.815741 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:47:59.815741 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:47:59.815741 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:47:59.815741 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:47:59.815741 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:47:59.815741 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:47:59.816817 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 15:47:59.816817 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 15:47:59.816817 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 15:47:59.816817 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Jun 23 15:47:59.818918 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 317/616] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Jun 23 15:47:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:47:59.824623 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97c1ff7-8256-4f06-a439-51372930ee0d None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:47:59.885271 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948018d4-70f8-48db-80d6-b8c1a9c471df None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:47:59.924374 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950af88e-f885-40cc-8001-0b2628236398 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:47:59.925589 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950af88e-f885-40cc-8001-0b2628236398 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:47:59.926904 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950af88e-f885-40cc-8001-0b2628236398 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:47:59.928133 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950af88e-f885-40cc-8001-0b2628236398 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:47:59.951610 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950af88e-f885-40cc-8001-0b2628236398 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:47:59.954277 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950af88e-f885-40cc-8001-0b2628236398 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:47:59.988857 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-950af88e-f885-40cc-8001-0b2628236398 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 15:47:59.988857 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:47:59.988857 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:47:59.988857 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:47:59.988857 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:47:59.988857 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:47:59.988857 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:47:59.988857 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:47:59.988857 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:47:59.988857 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:47:59.988857 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:47:59.988857 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:47:59.988857 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 15:47:59.988857 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 15:47:59.988857 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 15:47:59.990021 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 15:47:59.990021 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:47:59.990021 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:47:59.990021 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:47:59.990021 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:47:59.990021 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:47:59.990021 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:47:59.990021 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:47:59.990021 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Jun 23 15:47:59.990021 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:47:59.990021 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:47:59.990021 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:47:59.990021 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:47:59.990021 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:47:59.990021 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:47:59.990021 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:47:59.990021 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:47:59.991142 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 15:47:59.991142 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 15:47:59.991142 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 15:47:59.991142 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Jun 23 15:47:59.991142 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 300/617] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Jun 23 15:47:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 15:47:59.996813 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950af88e-f885-40cc-8001-0b2628236398 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:00.019443 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97c1ff7-8256-4f06-a439-51372930ee0d None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:00.060401 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0118e1-9343-4e03-87b1-918ceaf0cdb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:00.061415 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0118e1-9343-4e03-87b1-918ceaf0cdb0 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:00.061949 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0118e1-9343-4e03-87b1-918ceaf0cdb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:00.062378 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0118e1-9343-4e03-87b1-918ceaf0cdb0 None admin] PATH_INFO: `/v3/domains` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:00.063351 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0118e1-9343-4e03-87b1-918ceaf0cdb0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:48:00.063805 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0118e1-9343-4e03-87b1-918ceaf0cdb0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:00.065367 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0118e1-9343-4e03-87b1-918ceaf0cdb0 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:00.080874 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 318/618] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Jun 23 15:47:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 15:48:00.087437 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0118e1-9343-4e03-87b1-918ceaf0cdb0 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:00.098876 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:00.163407 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950af88e-f885-40cc-8001-0b2628236398 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:00.174782 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:00.232047 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b0edbb-d0b4-4559-b4ec-1769be3c0d26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:00.232784 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b0edbb-d0b4-4559-b4ec-1769be3c0d26 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:00.233074 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b0edbb-d0b4-4559-b4ec-1769be3c0d26 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:00.233354 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b0edbb-d0b4-4559-b4ec-1769be3c0d26 None admin] PATH_INFO: `/v3/domains` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:00.234090 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b0edbb-d0b4-4559-b4ec-1769be3c0d26 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:48:00.234406 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b0edbb-d0b4-4559-b4ec-1769be3c0d26 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:00.236056 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b0edbb-d0b4-4559-b4ec-1769be3c0d26 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:00.242889 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0118e1-9343-4e03-87b1-918ceaf0cdb0 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:00.256757 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 301/619] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Jun 23 15:47:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 15:48:00.267691 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b0edbb-d0b4-4559-b4ec-1769be3c0d26 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:00.287693 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ca0ec7-5555-4ace-a7d3-9be5ab904177 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:00.288768 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ca0ec7-5555-4ace-a7d3-9be5ab904177 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:00.289247 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ca0ec7-5555-4ace-a7d3-9be5ab904177 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:00.289669 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ca0ec7-5555-4ace-a7d3-9be5ab904177 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:00.299007 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ca0ec7-5555-4ace-a7d3-9be5ab904177 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:00.301090 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ca0ec7-5555-4ace-a7d3-9be5ab904177 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:00.317675 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-63ca0ec7-5555-4ace-a7d3-9be5ab904177 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 23 15:48:00.317675 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:48:00.317675 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:48:00.317675 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:48:00.317675 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:48:00.317675 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:48:00.317675 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:48:00.317675 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:48:00.317675 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:48:00.317675 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:48:00.317675 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:48:00.317675 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:48:00.317675 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 15:48:00.317675 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 15:48:00.317675 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 15:48:00.317675 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 15:48:00.319630 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:48:00.319630 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:48:00.319630 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 15:48:00.319630 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 15:48:00.319630 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 15:48:00.319630 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 15:48:00.319630 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:48:00.319630 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:48:00.319630 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:48:00.319630 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:48:00.319630 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:48:00.319630 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Jun 23 15:48:00.319630 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:48:00.319630 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:48:00.319630 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:48:00.319630 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:48:00.319630 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:48:00.319630 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:48:00.320750 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:48:00.320750 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:48:00.320750 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 15:48:00.320750 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 15:48:00.320750 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 15:48:00.320750 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 15:48:00.320750 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 15:48:00.320750 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 15:48:00.320750 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 23 15:48:00.320750 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Jun 23 15:48:00.323575 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 319/620] 10.222.0.46 () {56 vars in 1096 bytes} [Thu Jun 23 15:48:00 2022] GET /identity/v3/users/swift => generated 83 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:48:00.329367 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ca0ec7-5555-4ace-a7d3-9be5ab904177 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:00.446851 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ca0ec7-5555-4ace-a7d3-9be5ab904177 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:00.470314 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:00.488871 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92dfbf57-648b-4e85-af64-863f8f35005a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:00.489907 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dfbf57-648b-4e85-af64-863f8f35005a None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:00.490926 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dfbf57-648b-4e85-af64-863f8f35005a None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:00.491350 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dfbf57-648b-4e85-af64-863f8f35005a None admin] PATH_INFO: `/v3/users` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:00.492702 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92dfbf57-648b-4e85-af64-863f8f35005a None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:48:00.493221 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92dfbf57-648b-4e85-af64-863f8f35005a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:00.495358 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92dfbf57-648b-4e85-af64-863f8f35005a None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:00.519674 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 320/621] 10.222.0.46 () {56 vars in 1135 bytes} [Thu Jun 23 15:48:00 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 15:48:00.529233 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92dfbf57-648b-4e85-af64-863f8f35005a None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:00.569265 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b0edbb-d0b4-4559-b4ec-1769be3c0d26 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:00.645617 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92dfbf57-648b-4e85-af64-863f8f35005a None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:00.669225 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:00.687770 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ac24cb-723f-4d66-9ea0-5454141e8ebe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:00.688589 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ac24cb-723f-4d66-9ea0-5454141e8ebe None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:00.688872 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ac24cb-723f-4d66-9ea0-5454141e8ebe None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:00.689147 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ac24cb-723f-4d66-9ea0-5454141e8ebe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:00.700565 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ac24cb-723f-4d66-9ea0-5454141e8ebe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:00.702025 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ac24cb-723f-4d66-9ea0-5454141e8ebe None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:00.703405 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33830964-3d1c-440d-aff4-11aaac2f45da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:00.704342 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33830964-3d1c-440d-aff4-11aaac2f45da None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:00.704639 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33830964-3d1c-440d-aff4-11aaac2f45da None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:00.704925 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33830964-3d1c-440d-aff4-11aaac2f45da None admin] PATH_INFO: `/v3/users/nova` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:00.721657 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-39ac24cb-723f-4d66-9ea0-5454141e8ebe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 15:48:00.721657 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:48:00.721657 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:48:00.721657 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:48:00.721657 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:48:00.721657 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:48:00.721657 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:48:00.721657 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:48:00.721657 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:48:00.721657 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:48:00.721657 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:48:00.721657 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:48:00.721657 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 15:48:00.721657 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 15:48:00.721657 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 15:48:00.722626 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 15:48:00.722626 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:48:00.722626 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:48:00.722626 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:48:00.722626 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:48:00.722626 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:48:00.722626 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:48:00.722626 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:48:00.722626 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Jun 23 15:48:00.722626 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:48:00.722626 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:48:00.722626 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:48:00.722626 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:48:00.722626 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:48:00.722626 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:48:00.722626 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:48:00.722626 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:48:00.725173 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 15:48:00.725173 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 15:48:00.725173 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 15:48:00.725173 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Jun 23 15:48:00.725173 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 321/622] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Jun 23 15:48:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:48:00.729467 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ac24cb-723f-4d66-9ea0-5454141e8ebe None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:00.731571 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33830964-3d1c-440d-aff4-11aaac2f45da None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:00.733077 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33830964-3d1c-440d-aff4-11aaac2f45da None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:00.765338 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-33830964-3d1c-440d-aff4-11aaac2f45da None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 23 15:48:00.765338 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:48:00.765338 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:48:00.765338 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:48:00.765338 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:48:00.765338 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:48:00.765338 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:48:00.765338 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:48:00.765338 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:48:00.765338 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:48:00.765338 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:48:00.765338 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:48:00.765338 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 15:48:00.765338 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 15:48:00.765338 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 15:48:00.765338 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 15:48:00.766306 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:48:00.766306 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:48:00.766306 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 15:48:00.766306 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 15:48:00.766306 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 15:48:00.766306 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 15:48:00.766306 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:48:00.766306 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:48:00.766306 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:48:00.766306 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:48:00.766306 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:48:00.766306 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Jun 23 15:48:00.766306 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:48:00.766306 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:48:00.766306 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:48:00.766306 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:48:00.766306 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:48:00.766306 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:48:00.767286 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:48:00.767286 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:48:00.767286 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 15:48:00.767286 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 15:48:00.767286 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 15:48:00.767286 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 15:48:00.767286 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 15:48:00.767286 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 15:48:00.767286 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 23 15:48:00.767286 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Jun 23 15:48:00.767286 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 302/623] 10.222.0.46 () {56 vars in 1093 bytes} [Thu Jun 23 15:48:00 2022] GET /identity/v3/users/nova => generated 82 bytes in 499 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:48:00.771649 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:00.779851 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33830964-3d1c-440d-aff4-11aaac2f45da None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:00.861487 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ac24cb-723f-4d66-9ea0-5454141e8ebe None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:00.916447 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c656e966-e1e5-44ff-8fd4-d86ff48f3b8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:00.917553 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c656e966-e1e5-44ff-8fd4-d86ff48f3b8e None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:00.919066 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c656e966-e1e5-44ff-8fd4-d86ff48f3b8e None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:00.920068 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c656e966-e1e5-44ff-8fd4-d86ff48f3b8e None admin] PATH_INFO: `/v3/domains` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:00.928840 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c656e966-e1e5-44ff-8fd4-d86ff48f3b8e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:48:00.929445 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c656e966-e1e5-44ff-8fd4-d86ff48f3b8e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:00.931106 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c656e966-e1e5-44ff-8fd4-d86ff48f3b8e None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:00.958816 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 322/624] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Jun 23 15:48:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:48:00.967012 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c656e966-e1e5-44ff-8fd4-d86ff48f3b8e None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:01.023169 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33830964-3d1c-440d-aff4-11aaac2f45da None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:01.069602 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7d4a74-04d6-4dba-84ac-dc27ef343c35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:01.071688 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7d4a74-04d6-4dba-84ac-dc27ef343c35 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:01.072478 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7d4a74-04d6-4dba-84ac-dc27ef343c35 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:01.074169 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7d4a74-04d6-4dba-84ac-dc27ef343c35 None admin] PATH_INFO: `/v3/users` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:01.075494 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7d4a74-04d6-4dba-84ac-dc27ef343c35 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:48:01.076802 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7d4a74-04d6-4dba-84ac-dc27ef343c35 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:01.079368 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7d4a74-04d6-4dba-84ac-dc27ef343c35 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:01.121222 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 303/625] 10.222.0.46 () {56 vars in 1133 bytes} [Thu Jun 23 15:48:00 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 341 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:48:01.128131 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7d4a74-04d6-4dba-84ac-dc27ef343c35 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:01.203214 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:01.217637 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:01.220659 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c656e966-e1e5-44ff-8fd4-d86ff48f3b8e None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:01.282439 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5b8447-c04a-4831-b98a-d37f791379e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:01.283308 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5b8447-c04a-4831-b98a-d37f791379e3 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:01.285374 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5b8447-c04a-4831-b98a-d37f791379e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:01.285730 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5b8447-c04a-4831-b98a-d37f791379e3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:01.294424 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5b8447-c04a-4831-b98a-d37f791379e3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:01.295972 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5b8447-c04a-4831-b98a-d37f791379e3 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:01.296097 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7d4a74-04d6-4dba-84ac-dc27ef343c35 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:01.311481 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-6f5b8447-c04a-4831-b98a-d37f791379e3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 15:48:01.311481 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:48:01.311481 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:48:01.311481 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:48:01.311481 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:48:01.311481 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:48:01.311481 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:48:01.311481 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:48:01.311481 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:48:01.311481 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:48:01.311481 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:48:01.311481 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:48:01.311481 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 15:48:01.311481 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 15:48:01.311481 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 15:48:01.312436 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 15:48:01.312436 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:48:01.312436 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:48:01.312436 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:48:01.312436 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:48:01.312436 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:48:01.312436 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:48:01.312436 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:48:01.312436 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Jun 23 15:48:01.312436 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:48:01.312436 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:48:01.312436 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:48:01.312436 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:48:01.312436 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:48:01.312436 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:48:01.312436 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:48:01.312436 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:48:01.313316 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 15:48:01.313316 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 15:48:01.313316 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 15:48:01.313316 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 15:48:01.313316 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 15:48:01.313316 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 15:48:01.313316 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 15:48:01.313316 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Jun 23 15:48:01.313316 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 323/626] 10.222.0.46 () {56 vars in 1111 bytes} [Thu Jun 23 15:48:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 346 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:48:01.314115 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6518fe-a7d8-4e52-a49d-afe02c1612fd None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:01.314206 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6518fe-a7d8-4e52-a49d-afe02c1612fd None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:01.314451 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6518fe-a7d8-4e52-a49d-afe02c1612fd None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:01.320376 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 324/627] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jun 23 15:48:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 15:48:01.320376 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf6cea6-1173-4d10-863e-53c3b6c5767c None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:01.320376 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf6cea6-1173-4d10-863e-53c3b6c5767c None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:01.320376 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf6cea6-1173-4d10-863e-53c3b6c5767c None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:01.320376 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 325/628] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jun 23 15:48:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 15:48:01.320376 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf6cea6-1173-4d10-863e-53c3b6c5767c None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:01.343426 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbd9b7d-bd7c-4b6b-9ec8-408c4220ded6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:01.344098 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbd9b7d-bd7c-4b6b-9ec8-408c4220ded6 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:01.344384 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbd9b7d-bd7c-4b6b-9ec8-408c4220ded6 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:01.344658 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbd9b7d-bd7c-4b6b-9ec8-408c4220ded6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:01.421964 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbd9b7d-bd7c-4b6b-9ec8-408c4220ded6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:01.423456 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbd9b7d-bd7c-4b6b-9ec8-408c4220ded6 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:01.433288 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf6cea6-1173-4d10-863e-53c3b6c5767c None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:01.469326 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7d4895-8630-4388-86bd-27bc0969e998 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:01.470000 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7d4895-8630-4388-86bd-27bc0969e998 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:01.470281 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7d4895-8630-4388-86bd-27bc0969e998 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:01.470560 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7d4895-8630-4388-86bd-27bc0969e998 None admin] PATH_INFO: `/v3/projects` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:01.471384 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7d4895-8630-4388-86bd-27bc0969e998 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:48:01.471692 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7d4895-8630-4388-86bd-27bc0969e998 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:01.473152 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7d4895-8630-4388-86bd-27bc0969e998 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:01.474193 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:01.492137 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 326/629] 10.222.0.46 () {56 vars in 1148 bytes} [Thu Jun 23 15:48:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:48:01.493427 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dc0e0a-2947-4af1-a4da-5046cd456949 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:01.493715 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dc0e0a-2947-4af1-a4da-5046cd456949 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:01.493992 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dc0e0a-2947-4af1-a4da-5046cd456949 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:01.512500 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-55dc0e0a-2947-4af1-a4da-5046cd456949 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:48:01.513785 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-55dc0e0a-2947-4af1-a4da-5046cd456949 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:48:01.517458 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:01.544694 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 304/630] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Jun 23 15:48:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 15:48:01.544694 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a41562-56fa-4c0a-a362-c10d4cb6c48d None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:01.544694 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a41562-56fa-4c0a-a362-c10d4cb6c48d None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:01.544694 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a41562-56fa-4c0a-a362-c10d4cb6c48d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:01.569711 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-48a41562-56fa-4c0a-a362-c10d4cb6c48d None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:48:01.572429 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-48a41562-56fa-4c0a-a362-c10d4cb6c48d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:48:01.630727 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 327/631] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jun 23 15:48:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 15:48:01.630727 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55dc0e0a-2947-4af1-a4da-5046cd456949 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:01.671857 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 305/632] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jun 23 15:48:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 15:48:01.672836 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a41562-56fa-4c0a-a362-c10d4cb6c48d None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:01.781696 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:01.786879 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55dc0e0a-2947-4af1-a4da-5046cd456949 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:01.824733 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a41562-56fa-4c0a-a362-c10d4cb6c48d None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:01.845039 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23a3746-b8f8-4dc8-9c59-a5c1367f3eb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:01.846033 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23a3746-b8f8-4dc8-9c59-a5c1367f3eb4 None admin] REQUEST_METHOD: `PUT` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:01.846386 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23a3746-b8f8-4dc8-9c59-a5c1367f3eb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:01.846751 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23a3746-b8f8-4dc8-9c59-a5c1367f3eb4 None admin] PATH_INFO: `/v3/projects/28ecdf5b474c44098c722f677d2a0b77/users/242bf02d14724fe4837fcce8730377f9/roles/bfa8ecadc3d5483e821420aa0f0fb41e` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:01.859862 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:01.865194 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23a3746-b8f8-4dc8-9c59-a5c1367f3eb4 None admin] RBAC: Authorizing `identity:create_grant(project_id=28ecdf5b474c44098c722f677d2a0b77, user_id=242bf02d14724fe4837fcce8730377f9, role_id=bfa8ecadc3d5483e821420aa0f0fb41e)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:01.866800 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2466765-3e4d-4294-99d1-581c21df08b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:01.866977 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23a3746-b8f8-4dc8-9c59-a5c1367f3eb4 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:01.867701 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2466765-3e4d-4294-99d1-581c21df08b6 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:01.868053 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2466765-3e4d-4294-99d1-581c21df08b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:01.868402 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2466765-3e4d-4294-99d1-581c21df08b6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:01.882278 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2466765-3e4d-4294-99d1-581c21df08b6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:01.889173 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 328/633] 10.222.0.46 () {58 vars in 1439 bytes} [Thu Jun 23 15:48:01 2022] PUT /identity/v3/projects/28ecdf5b474c44098c722f677d2a0b77/users/242bf02d14724fe4837fcce8730377f9/roles/bfa8ecadc3d5483e821420aa0f0fb41e => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 15:48:01.889173 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23a3746-b8f8-4dc8-9c59-a5c1367f3eb4 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:01.889430 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2466765-3e4d-4294-99d1-581c21df08b6 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:01.914529 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-b2466765-3e4d-4294-99d1-581c21df08b6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 15:48:01.914529 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:48:01.914529 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:48:01.914529 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:48:01.914529 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:48:01.914529 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:48:01.914529 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:48:01.914529 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:48:01.914529 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:48:01.914529 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:48:01.914529 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:48:01.914529 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:48:01.914529 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 15:48:01.914529 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 15:48:01.914529 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 15:48:01.916119 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 15:48:01.916119 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:48:01.916119 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:48:01.916119 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:48:01.916119 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:48:01.916119 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:48:01.916119 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:48:01.916119 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:48:01.916119 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Jun 23 15:48:01.916119 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:48:01.916119 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:48:01.916119 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:48:01.916119 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:48:01.916119 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:48:01.916119 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:48:01.916119 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:48:01.916119 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:48:01.917601 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 15:48:01.917601 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 15:48:01.917601 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 15:48:01.917601 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Jun 23 15:48:01.917601 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 306/634] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Jun 23 15:48:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:48:01.917601 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2466765-3e4d-4294-99d1-581c21df08b6 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:02.055726 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23a3746-b8f8-4dc8-9c59-a5c1367f3eb4 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:02.061885 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2466765-3e4d-4294-99d1-581c21df08b6 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:02.099033 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15078086-96f1-4929-bab4-648ac7923e6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:02.100449 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15078086-96f1-4929-bab4-648ac7923e6b None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:02.102715 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15078086-96f1-4929-bab4-648ac7923e6b None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:02.102715 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15078086-96f1-4929-bab4-648ac7923e6b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:02.108135 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:02.118782 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80792ac2-f70b-4e0c-8b4d-989cff19a293 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:02.119673 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80792ac2-f70b-4e0c-8b4d-989cff19a293 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:02.119937 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15078086-96f1-4929-bab4-648ac7923e6b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 15:48:02.120045 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80792ac2-f70b-4e0c-8b4d-989cff19a293 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:02.120250 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80792ac2-f70b-4e0c-8b4d-989cff19a293 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:02.129634 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-15078086-96f1-4929-bab4-648ac7923e6b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 23 15:48:02.129634 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:48:02.129634 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:48:02.129634 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:48:02.129634 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:48:02.129634 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:48:02.129634 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:48:02.129634 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:48:02.129634 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:48:02.129634 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:48:02.129634 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:48:02.129634 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:48:02.129634 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 15:48:02.129634 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 15:48:02.129634 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 15:48:02.129634 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 15:48:02.131144 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 15:48:02.131144 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 15:48:02.131144 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 15:48:02.131144 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 15:48:02.131144 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:48:02.131144 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:48:02.131144 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:48:02.131144 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:48:02.131144 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:48:02.131144 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:48:02.131144 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:48:02.131144 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Jun 23 15:48:02.131144 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:48:02.131144 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:48:02.131144 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:48:02.131144 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:48:02.132019 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:48:02.132019 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:48:02.132019 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:48:02.132019 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:48:02.132019 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 15:48:02.132019 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 15:48:02.132019 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 15:48:02.132019 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 15:48:02.132019 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 15:48:02.132019 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 15:48:02.132019 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 23 15:48:02.132019 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Jun 23 15:48:02.135603 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 329/635] 10.222.0.46 () {56 vars in 1102 bytes} [Thu Jun 23 15:48:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:48:02.135603 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15078086-96f1-4929-bab4-648ac7923e6b None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:02.140899 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80792ac2-f70b-4e0c-8b4d-989cff19a293 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 15:48:02.153370 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-80792ac2-f70b-4e0c-8b4d-989cff19a293 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 23 15:48:02.153370 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:48:02.153370 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:48:02.153370 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:48:02.153370 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:48:02.153370 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:48:02.153370 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:48:02.153370 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:48:02.153370 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:48:02.153370 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:48:02.153370 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:48:02.153370 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:48:02.153370 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 15:48:02.153370 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 15:48:02.153370 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 15:48:02.153370 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 15:48:02.154417 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 15:48:02.154417 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 15:48:02.154417 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 15:48:02.154417 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 15:48:02.154417 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:48:02.154417 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:48:02.154417 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:48:02.154417 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:48:02.154417 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:48:02.154417 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:48:02.154417 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:48:02.154417 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Jun 23 15:48:02.154417 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:48:02.154417 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:48:02.154417 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:48:02.154417 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:48:02.155303 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:48:02.155303 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:48:02.155303 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:48:02.155303 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:48:02.155303 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 15:48:02.155303 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 15:48:02.155303 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 15:48:02.155303 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 15:48:02.155303 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 15:48:02.155303 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 15:48:02.155303 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 23 15:48:02.155303 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Jun 23 15:48:02.159981 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 307/636] 10.222.0.46 () {56 vars in 1102 bytes} [Thu Jun 23 15:48:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:48:02.161224 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80792ac2-f70b-4e0c-8b4d-989cff19a293 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:02.213448 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:02.277348 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15078086-96f1-4929-bab4-648ac7923e6b None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:02.314445 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80792ac2-f70b-4e0c-8b4d-989cff19a293 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:02.321130 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a53533-d0b7-4fba-94f2-994c9bf06012 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:02.322063 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a53533-d0b7-4fba-94f2-994c9bf06012 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:02.322634 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a53533-d0b7-4fba-94f2-994c9bf06012 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:02.323082 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a53533-d0b7-4fba-94f2-994c9bf06012 None admin] PATH_INFO: `/v3/domains` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:02.323982 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a53533-d0b7-4fba-94f2-994c9bf06012 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:48:02.324435 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a53533-d0b7-4fba-94f2-994c9bf06012 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:02.326194 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a53533-d0b7-4fba-94f2-994c9bf06012 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:02.340366 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 330/637] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Jun 23 15:48:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:48:02.341433 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a53533-d0b7-4fba-94f2-994c9bf06012 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:02.356213 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761afca9-3301-469d-9b63-2c292a32d767 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:02.357049 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761afca9-3301-469d-9b63-2c292a32d767 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:02.357338 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761afca9-3301-469d-9b63-2c292a32d767 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:02.357618 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761afca9-3301-469d-9b63-2c292a32d767 None admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:02.358374 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761afca9-3301-469d-9b63-2c292a32d767 None admin] RBAC: Adding query filter params (name=service) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:48:02.358726 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761afca9-3301-469d-9b63-2c292a32d767 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:02.360249 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761afca9-3301-469d-9b63-2c292a32d767 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:02.374825 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 308/638] 10.222.0.46 () {56 vars in 1103 bytes} [Thu Jun 23 15:48:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:48:02.376372 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4d2415-19ab-4abd-92bf-d4ce11edaabb None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:02.376653 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4d2415-19ab-4abd-92bf-d4ce11edaabb None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:02.376931 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4d2415-19ab-4abd-92bf-d4ce11edaabb None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:02.378494 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 309/639] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jun 23 15:48:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 15:48:02.379139 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4d2415-19ab-4abd-92bf-d4ce11edaabb None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:02.453458 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:02.486463 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a53533-d0b7-4fba-94f2-994c9bf06012 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:02.503599 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:02.510160 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4d2415-19ab-4abd-92bf-d4ce11edaabb None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:02.533143 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6baf9ad9-ab13-4fd0-881f-de19d356a1fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:02.533812 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6baf9ad9-ab13-4fd0-881f-de19d356a1fa None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:02.534100 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6baf9ad9-ab13-4fd0-881f-de19d356a1fa None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:02.534380 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6baf9ad9-ab13-4fd0-881f-de19d356a1fa None admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:02.535184 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6baf9ad9-ab13-4fd0-881f-de19d356a1fa None admin] RBAC: Adding query filter params (name=service) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:48:02.535804 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6baf9ad9-ab13-4fd0-881f-de19d356a1fa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:02.536960 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6baf9ad9-ab13-4fd0-881f-de19d356a1fa None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:02.545766 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 331/640] 10.222.0.46 () {56 vars in 1103 bytes} [Thu Jun 23 15:48:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:48:02.554667 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6baf9ad9-ab13-4fd0-881f-de19d356a1fa None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:02.559087 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc950091-d8f4-471a-bedb-82aa0d009df0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:02.559908 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc950091-d8f4-471a-bedb-82aa0d009df0 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:02.560190 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc950091-d8f4-471a-bedb-82aa0d009df0 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:02.560473 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc950091-d8f4-471a-bedb-82aa0d009df0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:02.567983 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc950091-d8f4-471a-bedb-82aa0d009df0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:02.570050 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc950091-d8f4-471a-bedb-82aa0d009df0 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:02.584867 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-bc950091-d8f4-471a-bedb-82aa0d009df0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 15:48:02.584867 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:48:02.584867 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:48:02.584867 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:48:02.584867 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:48:02.584867 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:48:02.584867 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:48:02.584867 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:48:02.584867 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:48:02.584867 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:48:02.584867 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:48:02.584867 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:48:02.584867 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 15:48:02.584867 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 15:48:02.584867 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 15:48:02.585945 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 15:48:02.585945 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:48:02.585945 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:48:02.585945 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:48:02.585945 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:48:02.585945 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:48:02.585945 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:48:02.585945 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:48:02.585945 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Jun 23 15:48:02.585945 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:48:02.585945 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:48:02.585945 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:48:02.585945 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:48:02.585945 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:48:02.585945 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:48:02.585945 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:48:02.585945 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:48:02.586907 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 15:48:02.586907 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 15:48:02.586907 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 15:48:02.586907 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 15:48:02.586907 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 15:48:02.586907 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 15:48:02.586907 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 15:48:02.586907 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Jun 23 15:48:02.586907 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 310/641] 10.222.0.46 () {56 vars in 1111 bytes} [Thu Jun 23 15:48:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:48:02.587407 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19695e7-de57-4f45-a059-cc463637de42 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:02.587714 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19695e7-de57-4f45-a059-cc463637de42 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:02.587965 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19695e7-de57-4f45-a059-cc463637de42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:02.607596 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-a19695e7-de57-4f45-a059-cc463637de42 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:48:02.608719 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-a19695e7-de57-4f45-a059-cc463637de42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:48:02.699685 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 311/642] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jun 23 15:48:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 15:48:02.700779 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19695e7-de57-4f45-a059-cc463637de42 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:02.718305 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6baf9ad9-ab13-4fd0-881f-de19d356a1fa None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:02.760113 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:02.767362 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da04eec-1d6c-419e-b957-60b91d05b444 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:02.768740 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da04eec-1d6c-419e-b957-60b91d05b444 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:02.769084 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da04eec-1d6c-419e-b957-60b91d05b444 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:02.769370 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da04eec-1d6c-419e-b957-60b91d05b444 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:02.782487 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da04eec-1d6c-419e-b957-60b91d05b444 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:02.784090 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da04eec-1d6c-419e-b957-60b91d05b444 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:02.807545 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-5da04eec-1d6c-419e-b957-60b91d05b444 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 15:48:02.807545 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:48:02.807545 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:48:02.807545 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:48:02.807545 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:48:02.807545 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:48:02.807545 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:48:02.807545 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:48:02.807545 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:48:02.807545 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:48:02.807545 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:48:02.807545 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:48:02.807545 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 15:48:02.807545 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 15:48:02.807545 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 15:48:02.808835 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 15:48:02.808835 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:48:02.808835 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:48:02.808835 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:48:02.808835 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:48:02.808835 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:48:02.808835 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:48:02.808835 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:48:02.808835 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Jun 23 15:48:02.808835 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:48:02.808835 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:48:02.808835 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:48:02.808835 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:48:02.808835 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:48:02.808835 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:48:02.808835 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:48:02.808835 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:48:02.809841 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 15:48:02.809841 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 15:48:02.809841 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 15:48:02.809841 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Jun 23 15:48:02.809841 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 332/643] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Jun 23 15:48:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:48:02.811114 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da04eec-1d6c-419e-b957-60b91d05b444 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:02.834660 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19695e7-de57-4f45-a059-cc463637de42 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:02.841858 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:02.873272 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e840c2ec-ef05-424a-80c6-5f766709a8e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:02.874098 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e840c2ec-ef05-424a-80c6-5f766709a8e4 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:02.874496 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e840c2ec-ef05-424a-80c6-5f766709a8e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:02.874795 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e840c2ec-ef05-424a-80c6-5f766709a8e4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:02.892237 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e840c2ec-ef05-424a-80c6-5f766709a8e4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:02.893890 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e840c2ec-ef05-424a-80c6-5f766709a8e4 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:02.914873 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-e840c2ec-ef05-424a-80c6-5f766709a8e4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 15:48:02.914873 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:48:02.914873 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:48:02.914873 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:48:02.914873 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:48:02.914873 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:48:02.914873 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:48:02.914873 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:48:02.914873 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:48:02.914873 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:48:02.914873 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:48:02.914873 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:48:02.914873 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 15:48:02.914873 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 15:48:02.914873 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 15:48:02.915866 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 15:48:02.915866 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:48:02.915866 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:48:02.915866 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:48:02.915866 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:48:02.915866 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:48:02.915866 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:48:02.915866 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:48:02.915866 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Jun 23 15:48:02.915866 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:48:02.915866 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:48:02.915866 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:48:02.915866 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:48:02.915866 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:48:02.915866 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:48:02.915866 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:48:02.915866 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:48:02.921940 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 15:48:02.921940 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 15:48:02.921940 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 15:48:02.921940 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Jun 23 15:48:02.921940 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 312/644] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Jun 23 15:48:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:48:02.921940 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e840c2ec-ef05-424a-80c6-5f766709a8e4 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:02.936977 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da04eec-1d6c-419e-b957-60b91d05b444 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:02.972432 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a85509-bf30-47ac-bcea-d5f90d753800 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:02.973102 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a85509-bf30-47ac-bcea-d5f90d753800 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:02.973391 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a85509-bf30-47ac-bcea-d5f90d753800 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:02.973668 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a85509-bf30-47ac-bcea-d5f90d753800 None admin] PATH_INFO: `/v3/projects` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:02.975000 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a85509-bf30-47ac-bcea-d5f90d753800 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:48:02.975315 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a85509-bf30-47ac-bcea-d5f90d753800 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:02.976760 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a85509-bf30-47ac-bcea-d5f90d753800 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:02.989196 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 333/645] 10.222.0.46 () {56 vars in 1148 bytes} [Thu Jun 23 15:48:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:48:02.990833 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a85509-bf30-47ac-bcea-d5f90d753800 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:03.026789 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:03.037487 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e840c2ec-ef05-424a-80c6-5f766709a8e4 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:03.083790 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fead6f-fe72-478f-b1df-babcf9f7558d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:03.084628 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fead6f-fe72-478f-b1df-babcf9f7558d None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:03.084918 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fead6f-fe72-478f-b1df-babcf9f7558d None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:03.085206 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fead6f-fe72-478f-b1df-babcf9f7558d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:03.105701 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fead6f-fe72-478f-b1df-babcf9f7558d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 15:48:03.110288 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:03.113606 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-a0fead6f-fe72-478f-b1df-babcf9f7558d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 23 15:48:03.113606 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:48:03.113606 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:48:03.113606 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:48:03.113606 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:48:03.113606 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:48:03.113606 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:48:03.113606 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:48:03.113606 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:48:03.113606 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:48:03.113606 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:48:03.113606 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:48:03.113606 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 15:48:03.113606 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 15:48:03.113606 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 15:48:03.113606 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 15:48:03.114770 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 15:48:03.114770 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 15:48:03.114770 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 15:48:03.114770 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 15:48:03.114770 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:48:03.114770 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:48:03.114770 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:48:03.114770 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:48:03.114770 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:48:03.114770 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:48:03.114770 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:48:03.114770 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Jun 23 15:48:03.114770 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:48:03.114770 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:48:03.114770 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:48:03.114770 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:48:03.116261 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:48:03.116261 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:48:03.116261 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:48:03.116261 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:48:03.116261 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 15:48:03.116261 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 15:48:03.116261 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 15:48:03.116261 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 15:48:03.116261 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 15:48:03.116261 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 15:48:03.116261 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 23 15:48:03.116261 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Jun 23 15:48:03.116261 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 313/646] 10.222.0.46 () {56 vars in 1102 bytes} [Thu Jun 23 15:48:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:48:03.116970 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fead6f-fe72-478f-b1df-babcf9f7558d None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:03.138610 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a85509-bf30-47ac-bcea-d5f90d753800 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:03.175336 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a558d8-4abc-45e7-a6df-ce332d5f3991 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:03.176351 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a558d8-4abc-45e7-a6df-ce332d5f3991 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:03.176817 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a558d8-4abc-45e7-a6df-ce332d5f3991 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:03.177301 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a558d8-4abc-45e7-a6df-ce332d5f3991 None admin] PATH_INFO: `/v3/domains` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:03.178771 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a558d8-4abc-45e7-a6df-ce332d5f3991 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:48:03.179296 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a558d8-4abc-45e7-a6df-ce332d5f3991 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:03.180741 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a558d8-4abc-45e7-a6df-ce332d5f3991 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:03.197667 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 334/647] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Jun 23 15:48:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:48:03.197667 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a558d8-4abc-45e7-a6df-ce332d5f3991 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:03.223593 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fead6f-fe72-478f-b1df-babcf9f7558d None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:03.259178 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b940cdf-9540-4d30-80aa-aad875f3f03b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:03.259948 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b940cdf-9540-4d30-80aa-aad875f3f03b None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:03.260304 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b940cdf-9540-4d30-80aa-aad875f3f03b None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:03.260651 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b940cdf-9540-4d30-80aa-aad875f3f03b None admin] PATH_INFO: `/v3/domains` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:03.261379 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b940cdf-9540-4d30-80aa-aad875f3f03b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:48:03.261685 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b940cdf-9540-4d30-80aa-aad875f3f03b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:03.263098 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b940cdf-9540-4d30-80aa-aad875f3f03b None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:03.275819 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 314/648] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Jun 23 15:48:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:48:03.280729 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b940cdf-9540-4d30-80aa-aad875f3f03b None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:03.296208 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:03.308940 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a558d8-4abc-45e7-a6df-ce332d5f3991 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:03.361952 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59f33ed-b5c3-4f29-af8d-6178923685dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:03.362895 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59f33ed-b5c3-4f29-af8d-6178923685dc None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:03.363329 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59f33ed-b5c3-4f29-af8d-6178923685dc None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:03.363453 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59f33ed-b5c3-4f29-af8d-6178923685dc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:03.364449 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59f33ed-b5c3-4f29-af8d-6178923685dc None admin] RBAC: Adding query filter params (role.id=bfa8ecadc3d5483e821420aa0f0fb41e, scope.project.id=28ecdf5b474c44098c722f677d2a0b77, user.id=705115d19bab424d894bea8b1b48a7f0) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:48:03.364953 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59f33ed-b5c3-4f29-af8d-6178923685dc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:03.367485 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59f33ed-b5c3-4f29-af8d-6178923685dc None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:03.377578 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 335/649] 10.222.0.46 () {56 vars in 1374 bytes} [Thu Jun 23 15:48:03 2022] GET /identity/v3/role_assignments?user.id=705115d19bab424d894bea8b1b48a7f0&scope.project.id=28ecdf5b474c44098c722f677d2a0b77&role.id=bfa8ecadc3d5483e821420aa0f0fb41e => generated 608 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:48:03.378429 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59f33ed-b5c3-4f29-af8d-6178923685dc None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:03.398067 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:03.413753 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b940cdf-9540-4d30-80aa-aad875f3f03b None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:03.461658 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50488278-9d4c-42ca-84a8-f0b9f51d98c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:03.462985 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50488278-9d4c-42ca-84a8-f0b9f51d98c9 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:03.463459 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50488278-9d4c-42ca-84a8-f0b9f51d98c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:03.463959 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50488278-9d4c-42ca-84a8-f0b9f51d98c9 None admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:03.465272 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50488278-9d4c-42ca-84a8-f0b9f51d98c9 None admin] RBAC: Adding query filter params (name=service) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:48:03.465795 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50488278-9d4c-42ca-84a8-f0b9f51d98c9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:03.468196 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50488278-9d4c-42ca-84a8-f0b9f51d98c9 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:03.483334 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 315/650] 10.222.0.46 () {56 vars in 1103 bytes} [Thu Jun 23 15:48:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:48:03.485353 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50488278-9d4c-42ca-84a8-f0b9f51d98c9 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:03.509358 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59f33ed-b5c3-4f29-af8d-6178923685dc None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:03.556435 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2829cfef-badd-4dfa-be81-9cd52657bd00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:03.557276 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2829cfef-badd-4dfa-be81-9cd52657bd00 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:03.557561 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2829cfef-badd-4dfa-be81-9cd52657bd00 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:03.557840 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2829cfef-badd-4dfa-be81-9cd52657bd00 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:03.566624 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2829cfef-badd-4dfa-be81-9cd52657bd00 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:03.568200 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2829cfef-badd-4dfa-be81-9cd52657bd00 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:03.585198 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-2829cfef-badd-4dfa-be81-9cd52657bd00 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 23 15:48:03.585198 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:48:03.585198 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:48:03.585198 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:48:03.585198 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:48:03.585198 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:48:03.585198 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:48:03.585198 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:48:03.585198 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:48:03.585198 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:48:03.585198 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:48:03.585198 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:48:03.585198 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 15:48:03.585198 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 15:48:03.585198 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 15:48:03.585198 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 15:48:03.586147 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:48:03.586147 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:48:03.586147 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 15:48:03.586147 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 15:48:03.586147 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 15:48:03.586147 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 15:48:03.586147 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:48:03.586147 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:48:03.586147 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:48:03.586147 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:48:03.586147 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:48:03.586147 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Jun 23 15:48:03.586147 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:48:03.586147 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:48:03.586147 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:48:03.586147 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:48:03.586147 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:48:03.586147 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:48:03.587124 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:48:03.587124 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:48:03.587124 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 15:48:03.587124 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 15:48:03.587124 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 15:48:03.587124 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 15:48:03.587124 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 15:48:03.587124 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 15:48:03.587124 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 23 15:48:03.587124 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Jun 23 15:48:03.587124 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 336/651] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Jun 23 15:48:03 2022] GET /identity/v3/users/cinder => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:48:03.591208 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2829cfef-badd-4dfa-be81-9cd52657bd00 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:03.604104 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:03.639156 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50488278-9d4c-42ca-84a8-f0b9f51d98c9 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:03.684147 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545917f0-5d5e-40f4-be4f-a73f66d09fe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:03.684761 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:03.684872 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545917f0-5d5e-40f4-be4f-a73f66d09fe4 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:03.685167 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545917f0-5d5e-40f4-be4f-a73f66d09fe4 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:03.685451 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545917f0-5d5e-40f4-be4f-a73f66d09fe4 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:03.695437 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545917f0-5d5e-40f4-be4f-a73f66d09fe4 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:03.696891 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545917f0-5d5e-40f4-be4f-a73f66d09fe4 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:03.713558 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-545917f0-5d5e-40f4-be4f-a73f66d09fe4 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 23 15:48:03.713558 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:48:03.713558 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:48:03.713558 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:48:03.713558 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:48:03.713558 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:48:03.713558 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:48:03.713558 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:48:03.713558 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:48:03.713558 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:48:03.713558 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:48:03.713558 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:48:03.713558 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 15:48:03.713558 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 15:48:03.713558 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 15:48:03.713558 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 15:48:03.714563 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:48:03.714563 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:48:03.714563 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 15:48:03.714563 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 15:48:03.714563 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 15:48:03.714563 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 15:48:03.714563 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:48:03.714563 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:48:03.714563 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:48:03.714563 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:48:03.714563 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:48:03.714563 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Jun 23 15:48:03.714563 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:48:03.714563 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:48:03.714563 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:48:03.714563 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:48:03.714563 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:48:03.714563 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:48:03.715558 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:48:03.715558 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:48:03.715558 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 15:48:03.715558 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 15:48:03.715558 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 15:48:03.715558 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 15:48:03.715558 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 15:48:03.715558 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 15:48:03.715558 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 23 15:48:03.715558 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Jun 23 15:48:03.718200 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 316/652] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Jun 23 15:48:03 2022] GET /identity/v3/users/glance => generated 84 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:48:03.723436 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545917f0-5d5e-40f4-be4f-a73f66d09fe4 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:03.729185 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2829cfef-badd-4dfa-be81-9cd52657bd00 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:03.766904 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f8788f-2da7-4817-bb87-8e165c1f9a46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:03.767527 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f8788f-2da7-4817-bb87-8e165c1f9a46 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:03.767815 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f8788f-2da7-4817-bb87-8e165c1f9a46 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:03.768091 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f8788f-2da7-4817-bb87-8e165c1f9a46 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:03.780477 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f8788f-2da7-4817-bb87-8e165c1f9a46 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:03.782349 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f8788f-2da7-4817-bb87-8e165c1f9a46 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:03.800901 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-96f8788f-2da7-4817-bb87-8e165c1f9a46 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 15:48:03.800901 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:48:03.800901 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:48:03.800901 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:48:03.800901 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:48:03.800901 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:48:03.800901 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:48:03.800901 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:48:03.800901 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:48:03.800901 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:48:03.800901 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:48:03.800901 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:48:03.800901 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 15:48:03.800901 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 15:48:03.800901 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 15:48:03.801933 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 15:48:03.801933 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:48:03.801933 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:48:03.801933 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:48:03.801933 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:48:03.801933 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:48:03.801933 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:48:03.801933 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:48:03.801933 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Jun 23 15:48:03.801933 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:48:03.801933 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:48:03.801933 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:48:03.801933 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:48:03.801933 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:48:03.801933 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:48:03.801933 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:48:03.801933 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:48:03.802895 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 15:48:03.802895 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 15:48:03.802895 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 15:48:03.802895 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Jun 23 15:48:03.802895 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 337/653] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Jun 23 15:48:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:48:03.807772 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f8788f-2da7-4817-bb87-8e165c1f9a46 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:03.854754 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545917f0-5d5e-40f4-be4f-a73f66d09fe4 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:03.906016 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac1829b-4805-45b7-bebb-8975bab22ba5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:03.907265 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac1829b-4805-45b7-bebb-8975bab22ba5 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:03.907743 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac1829b-4805-45b7-bebb-8975bab22ba5 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:03.908669 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac1829b-4805-45b7-bebb-8975bab22ba5 None admin] PATH_INFO: `/v3/users` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:03.909966 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac1829b-4805-45b7-bebb-8975bab22ba5 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:48:03.910438 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac1829b-4805-45b7-bebb-8975bab22ba5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:03.912922 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac1829b-4805-45b7-bebb-8975bab22ba5 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:03.916791 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:03.929458 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f8788f-2da7-4817-bb87-8e165c1f9a46 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:03.938505 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 317/654] 10.222.0.46 () {56 vars in 1137 bytes} [Thu Jun 23 15:48:03 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:48:03.939621 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac1829b-4805-45b7-bebb-8975bab22ba5 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:03.959324 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:03.979605 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8023bf60-5838-44c1-a781-f79c1ce0d899 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:03.980525 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8023bf60-5838-44c1-a781-f79c1ce0d899 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:03.981086 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8023bf60-5838-44c1-a781-f79c1ce0d899 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:03.981537 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8023bf60-5838-44c1-a781-f79c1ce0d899 None admin] PATH_INFO: `/v3/users` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:03.983381 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8023bf60-5838-44c1-a781-f79c1ce0d899 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:48:03.983848 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8023bf60-5838-44c1-a781-f79c1ce0d899 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:03.985481 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8023bf60-5838-44c1-a781-f79c1ce0d899 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:04.010526 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 338/655] 10.222.0.46 () {56 vars in 1137 bytes} [Thu Jun 23 15:48:03 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:48:04.011627 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8023bf60-5838-44c1-a781-f79c1ce0d899 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:04.059886 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac1829b-4805-45b7-bebb-8975bab22ba5 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:04.097364 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8b3903-5312-453b-9308-1bb59b614b96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:04.098157 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8b3903-5312-453b-9308-1bb59b614b96 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:04.098446 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8b3903-5312-453b-9308-1bb59b614b96 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:04.098750 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8b3903-5312-453b-9308-1bb59b614b96 None admin] PATH_INFO: `/v3/domains` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:04.099636 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8b3903-5312-453b-9308-1bb59b614b96 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:48:04.099947 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8b3903-5312-453b-9308-1bb59b614b96 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:04.101424 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8b3903-5312-453b-9308-1bb59b614b96 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:04.118727 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 318/656] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Jun 23 15:48:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:48:04.118727 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8b3903-5312-453b-9308-1bb59b614b96 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:04.137885 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8023bf60-5838-44c1-a781-f79c1ce0d899 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:04.174581 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:04.181219 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1355f1-d481-43f4-8e31-1656435fc774 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:04.181905 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1355f1-d481-43f4-8e31-1656435fc774 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:04.182193 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1355f1-d481-43f4-8e31-1656435fc774 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:04.182473 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1355f1-d481-43f4-8e31-1656435fc774 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:04.227750 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8b3903-5312-453b-9308-1bb59b614b96 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:04.238910 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:04.264083 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce84af8f-4dde-4a87-b36c-0546e48248ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:04.265151 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce84af8f-4dde-4a87-b36c-0546e48248ef None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:04.265332 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1355f1-d481-43f4-8e31-1656435fc774 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:04.265825 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce84af8f-4dde-4a87-b36c-0546e48248ef None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:04.266307 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce84af8f-4dde-4a87-b36c-0546e48248ef None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:04.266819 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1355f1-d481-43f4-8e31-1656435fc774 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:04.340624 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce84af8f-4dde-4a87-b36c-0546e48248ef None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:04.342071 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce84af8f-4dde-4a87-b36c-0546e48248ef None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:04.382531 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 339/657] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Jun 23 15:48:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 15:48:04.383994 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2039b8-7ae6-49e9-a5a2-10b46c88b0e5 None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:04.384279 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2039b8-7ae6-49e9-a5a2-10b46c88b0e5 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:04.384555 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2039b8-7ae6-49e9-a5a2-10b46c88b0e5 None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:04.386054 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 340/658] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jun 23 15:48:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 15:48:04.386615 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2039b8-7ae6-49e9-a5a2-10b46c88b0e5 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:04.406841 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:04.467213 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 319/659] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Jun 23 15:48:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 15:48:04.468606 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce84af8f-4dde-4a87-b36c-0546e48248ef None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:04.486608 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:04.512579 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2039b8-7ae6-49e9-a5a2-10b46c88b0e5 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:04.561407 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3af1cff-ed50-4107-a670-42ff83854952 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:04.562361 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3af1cff-ed50-4107-a670-42ff83854952 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:04.562645 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3af1cff-ed50-4107-a670-42ff83854952 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:04.562981 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3af1cff-ed50-4107-a670-42ff83854952 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:04.573953 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3af1cff-ed50-4107-a670-42ff83854952 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:04.573953 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3af1cff-ed50-4107-a670-42ff83854952 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:04.591967 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-e3af1cff-ed50-4107-a670-42ff83854952 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 23 15:48:04.591967 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:48:04.591967 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:48:04.591967 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:48:04.591967 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:48:04.591967 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:48:04.591967 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:48:04.591967 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:48:04.591967 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:48:04.591967 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:48:04.591967 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:48:04.591967 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:48:04.591967 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 15:48:04.591967 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 15:48:04.591967 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 15:48:04.591967 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 15:48:04.593113 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:48:04.593113 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:48:04.593113 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 15:48:04.593113 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 15:48:04.593113 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 15:48:04.593113 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 15:48:04.593113 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:48:04.593113 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:48:04.593113 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:48:04.593113 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:48:04.593113 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:48:04.593113 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Jun 23 15:48:04.593113 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:48:04.593113 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:48:04.593113 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:48:04.593113 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:48:04.593113 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:48:04.593113 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:48:04.594054 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:48:04.594054 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:48:04.594054 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 15:48:04.594054 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 15:48:04.594054 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 15:48:04.594054 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 15:48:04.594054 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 15:48:04.594054 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 15:48:04.594054 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 23 15:48:04.594054 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Jun 23 15:48:04.594054 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 341/660] 10.222.0.46 () {56 vars in 1102 bytes} [Thu Jun 23 15:48:04 2022] GET /identity/v3/users/neutron => generated 85 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:48:04.594830 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d96f11-d00a-476c-8816-2470b0b8d652 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:04.594830 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d96f11-d00a-476c-8816-2470b0b8d652 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:04.595487 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d96f11-d00a-476c-8816-2470b0b8d652 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:04.621924 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-86d96f11-d00a-476c-8816-2470b0b8d652 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:48:04.623553 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-86d96f11-d00a-476c-8816-2470b0b8d652 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:48:04.633658 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce84af8f-4dde-4a87-b36c-0546e48248ef None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:04.692112 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb9d01b-0b2c-430e-b5db-252380225469 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:04.692951 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb9d01b-0b2c-430e-b5db-252380225469 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:04.693248 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb9d01b-0b2c-430e-b5db-252380225469 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:04.693526 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb9d01b-0b2c-430e-b5db-252380225469 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:04.704608 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb9d01b-0b2c-430e-b5db-252380225469 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:04.706121 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb9d01b-0b2c-430e-b5db-252380225469 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:04.714780 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 342/661] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jun 23 15:48:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 15:48:04.714780 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d96f11-d00a-476c-8816-2470b0b8d652 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:04.726415 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-ebb9d01b-0b2c-430e-b5db-252380225469 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 15:48:04.726415 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:48:04.726415 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:48:04.726415 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:48:04.726415 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:48:04.726415 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:48:04.726415 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:48:04.726415 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:48:04.726415 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:48:04.726415 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:48:04.726415 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:48:04.726415 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:48:04.726415 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 15:48:04.726415 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 15:48:04.726415 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 15:48:04.727532 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 15:48:04.727532 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:48:04.727532 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:48:04.727532 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:48:04.727532 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:48:04.727532 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:48:04.727532 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:48:04.727532 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:48:04.727532 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Jun 23 15:48:04.727532 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:48:04.727532 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:48:04.727532 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:48:04.727532 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:48:04.727532 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:48:04.727532 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:48:04.727532 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:48:04.727532 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:48:04.728551 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 15:48:04.728551 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 15:48:04.728551 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 15:48:04.728551 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Jun 23 15:48:04.728551 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 320/662] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Jun 23 15:48:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:48:04.728900 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb9d01b-0b2c-430e-b5db-252380225469 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:04.739878 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:04.806436 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:04.848280 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb9d01b-0b2c-430e-b5db-252380225469 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:04.848519 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d96f11-d00a-476c-8816-2470b0b8d652 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:04.884117 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85443504-18dc-427c-a421-10e73344e2cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:04.884550 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3867ff5b-f9b3-4685-88c1-1b93f50b8980 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:04.884806 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85443504-18dc-427c-a421-10e73344e2cb None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:04.885162 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85443504-18dc-427c-a421-10e73344e2cb None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:04.885349 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3867ff5b-f9b3-4685-88c1-1b93f50b8980 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:04.885467 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85443504-18dc-427c-a421-10e73344e2cb None admin] PATH_INFO: `/v3/users` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:04.885631 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3867ff5b-f9b3-4685-88c1-1b93f50b8980 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:04.885904 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3867ff5b-f9b3-4685-88c1-1b93f50b8980 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:04.886209 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85443504-18dc-427c-a421-10e73344e2cb None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:48:04.886523 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85443504-18dc-427c-a421-10e73344e2cb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:04.888019 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85443504-18dc-427c-a421-10e73344e2cb None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:04.899460 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3867ff5b-f9b3-4685-88c1-1b93f50b8980 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:04.901833 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3867ff5b-f9b3-4685-88c1-1b93f50b8980 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:04.912981 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 321/663] 10.222.0.46 () {56 vars in 1139 bytes} [Thu Jun 23 15:48:04 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:48:04.914725 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85443504-18dc-427c-a421-10e73344e2cb None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:04.936241 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-3867ff5b-f9b3-4685-88c1-1b93f50b8980 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 15:48:04.936241 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:48:04.936241 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:48:04.936241 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:48:04.936241 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:48:04.936241 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:48:04.936241 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:48:04.936241 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:48:04.936241 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:48:04.936241 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:48:04.936241 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:48:04.936241 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:48:04.936241 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 15:48:04.936241 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 15:48:04.936241 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 15:48:04.937256 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 15:48:04.937256 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:48:04.937256 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:48:04.937256 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:48:04.937256 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:48:04.937256 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:48:04.937256 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:48:04.937256 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:48:04.937256 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Jun 23 15:48:04.937256 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:48:04.937256 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:48:04.937256 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:48:04.937256 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:48:04.937256 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:48:04.937256 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:48:04.937256 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:48:04.937256 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:48:04.938156 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 15:48:04.938156 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 15:48:04.938156 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 15:48:04.938156 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Jun 23 15:48:04.938156 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 343/664] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Jun 23 15:48:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:48:04.938451 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3867ff5b-f9b3-4685-88c1-1b93f50b8980 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:04.994957 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:05.048949 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85443504-18dc-427c-a421-10e73344e2cb None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:05.063640 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3867ff5b-f9b3-4685-88c1-1b93f50b8980 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:05.086423 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e91a466-26a6-4e4b-a035-ee7277191ba9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:05.087266 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e91a466-26a6-4e4b-a035-ee7277191ba9 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:05.087550 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e91a466-26a6-4e4b-a035-ee7277191ba9 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:05.087828 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e91a466-26a6-4e4b-a035-ee7277191ba9 None admin] PATH_INFO: `/v3/domains` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:05.088568 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e91a466-26a6-4e4b-a035-ee7277191ba9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:48:05.088883 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e91a466-26a6-4e4b-a035-ee7277191ba9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:05.090471 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e91a466-26a6-4e4b-a035-ee7277191ba9 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:05.093847 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:05.109763 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 322/665] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Jun 23 15:48:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:48:05.109763 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e91a466-26a6-4e4b-a035-ee7277191ba9 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:05.117085 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723567b8-0da2-49ec-b4b1-01f3e460ea50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:05.118454 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723567b8-0da2-49ec-b4b1-01f3e460ea50 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:05.119015 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723567b8-0da2-49ec-b4b1-01f3e460ea50 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:05.119525 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723567b8-0da2-49ec-b4b1-01f3e460ea50 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:05.138142 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723567b8-0da2-49ec-b4b1-01f3e460ea50 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 15:48:05.153584 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-723567b8-0da2-49ec-b4b1-01f3e460ea50 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 23 15:48:05.153584 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:48:05.153584 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:48:05.153584 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:48:05.153584 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:48:05.153584 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:48:05.153584 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:48:05.153584 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:48:05.153584 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:48:05.153584 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:48:05.153584 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:48:05.153584 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:48:05.153584 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 15:48:05.153584 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 15:48:05.153584 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 15:48:05.153584 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 15:48:05.154579 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 15:48:05.154579 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 15:48:05.154579 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 15:48:05.154579 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 15:48:05.154579 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:48:05.154579 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:48:05.154579 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:48:05.154579 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:48:05.154579 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:48:05.154579 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:48:05.154579 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:48:05.154579 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Jun 23 15:48:05.154579 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:48:05.154579 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:48:05.154579 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:48:05.154579 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:48:05.155449 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:48:05.155449 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:48:05.155449 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:48:05.155449 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:48:05.155449 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 15:48:05.155449 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 15:48:05.155449 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 15:48:05.155449 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 15:48:05.155449 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 15:48:05.155449 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 15:48:05.155449 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 23 15:48:05.155449 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Jun 23 15:48:05.160332 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 344/666] 10.222.0.46 () {56 vars in 1102 bytes} [Thu Jun 23 15:48:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:48:05.160332 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723567b8-0da2-49ec-b4b1-01f3e460ea50 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:05.244323 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e91a466-26a6-4e4b-a035-ee7277191ba9 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:05.274800 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723567b8-0da2-49ec-b4b1-01f3e460ea50 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:05.287466 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:05.301168 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5d273c-7965-42c8-a707-6ad4f08bc1f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:05.301831 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5d273c-7965-42c8-a707-6ad4f08bc1f2 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:05.302308 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5d273c-7965-42c8-a707-6ad4f08bc1f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:05.302466 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5d273c-7965-42c8-a707-6ad4f08bc1f2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:05.315608 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048fee11-48b5-4785-a657-260d8c02fbf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:05.316290 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048fee11-48b5-4785-a657-260d8c02fbf1 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:05.316607 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048fee11-48b5-4785-a657-260d8c02fbf1 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:05.316967 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048fee11-48b5-4785-a657-260d8c02fbf1 None admin] PATH_INFO: `/v3/domains` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:05.317703 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048fee11-48b5-4785-a657-260d8c02fbf1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:48:05.318626 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048fee11-48b5-4785-a657-260d8c02fbf1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:05.320099 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048fee11-48b5-4785-a657-260d8c02fbf1 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:05.332504 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 345/667] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Jun 23 15:48:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:48:05.335633 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048fee11-48b5-4785-a657-260d8c02fbf1 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:05.379255 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5d273c-7965-42c8-a707-6ad4f08bc1f2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:05.381393 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5d273c-7965-42c8-a707-6ad4f08bc1f2 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:05.387769 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:05.456996 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048fee11-48b5-4785-a657-260d8c02fbf1 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:05.500578 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 323/668] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Jun 23 15:48:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 15:48:05.501804 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5d273c-7965-42c8-a707-6ad4f08bc1f2 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:05.507296 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a2dad7-efd1-4511-918a-a794477d0bca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:05.508456 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a2dad7-efd1-4511-918a-a794477d0bca None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:05.508965 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a2dad7-efd1-4511-918a-a794477d0bca None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:05.509476 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a2dad7-efd1-4511-918a-a794477d0bca None admin] PATH_INFO: `/v3/projects/service` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:05.521482 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a2dad7-efd1-4511-918a-a794477d0bca None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:05.523004 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a2dad7-efd1-4511-918a-a794477d0bca None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:05.531925 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:05.539577 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-30a2dad7-efd1-4511-918a-a794477d0bca None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 15:48:05.539577 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:48:05.539577 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:48:05.539577 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:48:05.539577 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:48:05.539577 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:48:05.539577 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:48:05.539577 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:48:05.539577 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:48:05.539577 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:48:05.539577 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:48:05.539577 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:48:05.539577 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 15:48:05.539577 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 15:48:05.539577 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 15:48:05.540518 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 15:48:05.540518 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:48:05.540518 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:48:05.540518 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:48:05.540518 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:48:05.540518 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:48:05.540518 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:48:05.540518 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:48:05.540518 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Jun 23 15:48:05.540518 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:48:05.540518 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:48:05.540518 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:48:05.540518 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:48:05.540518 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:48:05.540518 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:48:05.540518 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:48:05.540518 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:48:05.541426 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 15:48:05.541426 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 15:48:05.541426 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 15:48:05.541426 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 15:48:05.541426 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 15:48:05.541426 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 15:48:05.541426 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 15:48:05.541426 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Jun 23 15:48:05.541426 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 346/669] 10.222.0.46 () {56 vars in 1111 bytes} [Thu Jun 23 15:48:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:48:05.541906 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a2dad7-efd1-4511-918a-a794477d0bca None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:05.648744 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5d273c-7965-42c8-a707-6ad4f08bc1f2 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:05.674336 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:05.678067 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a2dad7-efd1-4511-918a-a794477d0bca None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:05.696359 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f43cbc-fd11-403b-804b-56d86354bc5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:05.697053 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f43cbc-fd11-403b-804b-56d86354bc5e None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:05.697346 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f43cbc-fd11-403b-804b-56d86354bc5e None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:05.697634 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f43cbc-fd11-403b-804b-56d86354bc5e None admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:05.698397 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f43cbc-fd11-403b-804b-56d86354bc5e None admin] RBAC: Adding query filter params (name=service) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:48:05.698736 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f43cbc-fd11-403b-804b-56d86354bc5e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:05.700130 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f43cbc-fd11-403b-804b-56d86354bc5e None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:05.715871 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 324/670] 10.222.0.46 () {56 vars in 1103 bytes} [Thu Jun 23 15:48:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:48:05.715871 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f43cbc-fd11-403b-804b-56d86354bc5e None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:05.723815 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53c3c6b-0b2b-4471-a8a2-4e66a4f9367b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:05.725202 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53c3c6b-0b2b-4471-a8a2-4e66a4f9367b None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:05.725484 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53c3c6b-0b2b-4471-a8a2-4e66a4f9367b None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:05.725761 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53c3c6b-0b2b-4471-a8a2-4e66a4f9367b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:05.732714 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53c3c6b-0b2b-4471-a8a2-4e66a4f9367b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:05.734171 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53c3c6b-0b2b-4471-a8a2-4e66a4f9367b None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:05.748941 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-e53c3c6b-0b2b-4471-a8a2-4e66a4f9367b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 15:48:05.748941 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:48:05.748941 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:48:05.748941 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:48:05.748941 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:48:05.748941 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:48:05.748941 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:48:05.748941 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:48:05.748941 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:48:05.748941 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:48:05.748941 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:48:05.748941 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:48:05.748941 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 15:48:05.748941 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 15:48:05.748941 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 15:48:05.749925 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 15:48:05.749925 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:48:05.749925 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:48:05.749925 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:48:05.749925 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:48:05.749925 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:48:05.749925 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:48:05.749925 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:48:05.749925 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Jun 23 15:48:05.749925 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:48:05.749925 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:48:05.749925 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:48:05.749925 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:48:05.749925 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:48:05.749925 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:48:05.749925 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:48:05.749925 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:48:05.750868 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 15:48:05.750868 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 15:48:05.750868 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 15:48:05.750868 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 15:48:05.750868 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 15:48:05.750868 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 15:48:05.750868 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 15:48:05.750868 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Jun 23 15:48:05.750868 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 347/671] 10.222.0.46 () {56 vars in 1111 bytes} [Thu Jun 23 15:48:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:48:05.751349 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53c3c6b-0b2b-4471-a8a2-4e66a4f9367b None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:05.809655 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:05.826084 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f43cbc-fd11-403b-804b-56d86354bc5e None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:05.864416 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83166749-5505-4c19-8412-1d2b0a0419f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:05.865585 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83166749-5505-4c19-8412-1d2b0a0419f7 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:05.865865 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83166749-5505-4c19-8412-1d2b0a0419f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:05.866131 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83166749-5505-4c19-8412-1d2b0a0419f7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:05.878168 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53c3c6b-0b2b-4471-a8a2-4e66a4f9367b None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:05.881089 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83166749-5505-4c19-8412-1d2b0a0419f7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:05.882743 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83166749-5505-4c19-8412-1d2b0a0419f7 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:05.904473 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-83166749-5505-4c19-8412-1d2b0a0419f7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 15:48:05.904473 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:48:05.904473 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:48:05.904473 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:48:05.904473 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:48:05.904473 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:48:05.904473 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:48:05.904473 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:48:05.904473 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:48:05.904473 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:48:05.904473 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:48:05.904473 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:48:05.904473 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 15:48:05.904473 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 15:48:05.904473 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 15:48:05.905600 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 15:48:05.905600 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:48:05.905600 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:48:05.905600 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:48:05.905600 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:48:05.905600 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:48:05.905600 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:48:05.905600 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:48:05.905600 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Jun 23 15:48:05.905600 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:48:05.905600 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:48:05.905600 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:48:05.905600 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:48:05.905600 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:48:05.905600 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:48:05.905600 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:48:05.905600 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:48:05.906663 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 15:48:05.906663 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 15:48:05.906663 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 15:48:05.906663 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Jun 23 15:48:05.906663 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 325/672] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Jun 23 15:48:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:48:05.907119 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83166749-5505-4c19-8412-1d2b0a0419f7 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:05.925025 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7ab9d4-40a2-4967-9393-ee4bcf50f2a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:05.925695 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7ab9d4-40a2-4967-9393-ee4bcf50f2a9 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:05.925974 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7ab9d4-40a2-4967-9393-ee4bcf50f2a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:05.926247 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7ab9d4-40a2-4967-9393-ee4bcf50f2a9 None admin] PATH_INFO: `/v3/projects` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:05.927070 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7ab9d4-40a2-4967-9393-ee4bcf50f2a9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:48:05.927420 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7ab9d4-40a2-4967-9393-ee4bcf50f2a9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:05.928866 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7ab9d4-40a2-4967-9393-ee4bcf50f2a9 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:05.941691 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 348/673] 10.222.0.46 () {56 vars in 1148 bytes} [Thu Jun 23 15:48:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:48:05.943449 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc05420-5b51-421c-9598-c4f3506104d9 None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:05.943738 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc05420-5b51-421c-9598-c4f3506104d9 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:05.944020 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc05420-5b51-421c-9598-c4f3506104d9 None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:05.945264 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 349/674] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jun 23 15:48:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 15:48:05.945957 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc05420-5b51-421c-9598-c4f3506104d9 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:05.970281 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:06.027180 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83166749-5505-4c19-8412-1d2b0a0419f7 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:06.061197 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc05420-5b51-421c-9598-c4f3506104d9 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:06.067355 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b342e72-ae9a-4a9b-9f17-d58153b29219 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:06.068666 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b342e72-ae9a-4a9b-9f17-d58153b29219 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:06.068950 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b342e72-ae9a-4a9b-9f17-d58153b29219 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:06.069227 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b342e72-ae9a-4a9b-9f17-d58153b29219 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:06.076263 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:06.081721 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b342e72-ae9a-4a9b-9f17-d58153b29219 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:06.084256 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b342e72-ae9a-4a9b-9f17-d58153b29219 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:06.101467 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-9b342e72-ae9a-4a9b-9f17-d58153b29219 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 15:48:06.101467 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:48:06.101467 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:48:06.101467 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:48:06.101467 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:48:06.101467 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:48:06.101467 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:48:06.101467 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:48:06.101467 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:48:06.101467 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:48:06.101467 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:48:06.101467 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:48:06.101467 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 15:48:06.101467 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 15:48:06.101467 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 15:48:06.102405 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 15:48:06.102405 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:48:06.102405 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:48:06.102405 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:48:06.102405 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:48:06.102405 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:48:06.102405 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:48:06.102405 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:48:06.102405 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Jun 23 15:48:06.102405 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:48:06.102405 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:48:06.102405 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:48:06.102405 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:48:06.102405 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:48:06.102405 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:48:06.102405 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:48:06.102405 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:48:06.103520 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 15:48:06.103520 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 15:48:06.103520 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 15:48:06.103520 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Jun 23 15:48:06.103520 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 326/675] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Jun 23 15:48:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:48:06.104242 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b342e72-ae9a-4a9b-9f17-d58153b29219 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:06.109552 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471102f6-746f-41c5-92b9-c7f703e5ac97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:06.110229 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471102f6-746f-41c5-92b9-c7f703e5ac97 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:06.110507 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471102f6-746f-41c5-92b9-c7f703e5ac97 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:06.110817 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471102f6-746f-41c5-92b9-c7f703e5ac97 None admin] PATH_INFO: `/v3/projects` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:06.111700 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471102f6-746f-41c5-92b9-c7f703e5ac97 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:48:06.112013 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471102f6-746f-41c5-92b9-c7f703e5ac97 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:06.113904 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471102f6-746f-41c5-92b9-c7f703e5ac97 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:06.133855 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 350/676] 10.222.0.46 () {56 vars in 1148 bytes} [Thu Jun 23 15:48:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:48:06.134748 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471102f6-746f-41c5-92b9-c7f703e5ac97 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:06.225083 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b342e72-ae9a-4a9b-9f17-d58153b29219 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:06.244711 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:06.263049 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35238a3-2b5c-40dd-b3ef-58463ac9b03e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:06.266873 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471102f6-746f-41c5-92b9-c7f703e5ac97 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:06.271835 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35238a3-2b5c-40dd-b3ef-58463ac9b03e None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:06.272350 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35238a3-2b5c-40dd-b3ef-58463ac9b03e None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:06.272809 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35238a3-2b5c-40dd-b3ef-58463ac9b03e None admin] PATH_INFO: `/v3/domains` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:06.273579 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35238a3-2b5c-40dd-b3ef-58463ac9b03e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:48:06.273900 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35238a3-2b5c-40dd-b3ef-58463ac9b03e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:06.275636 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35238a3-2b5c-40dd-b3ef-58463ac9b03e None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:06.287948 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 327/677] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Jun 23 15:48:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:48:06.289309 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b7b584-d9cd-46be-b188-ecaaaf8e2625 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:06.289605 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b7b584-d9cd-46be-b188-ecaaaf8e2625 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:06.289886 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b7b584-d9cd-46be-b188-ecaaaf8e2625 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:06.307541 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-f1b7b584-d9cd-46be-b188-ecaaaf8e2625 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:48:06.308750 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-f1b7b584-d9cd-46be-b188-ecaaaf8e2625 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:48:06.310356 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c524ed1-7726-4c03-8407-e84487691a56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:06.311060 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c524ed1-7726-4c03-8407-e84487691a56 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:06.311393 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c524ed1-7726-4c03-8407-e84487691a56 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:06.311670 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c524ed1-7726-4c03-8407-e84487691a56 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:06.312473 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c524ed1-7726-4c03-8407-e84487691a56 None admin] RBAC: Adding query filter params (role.id=bfa8ecadc3d5483e821420aa0f0fb41e, scope.project.id=28ecdf5b474c44098c722f677d2a0b77, user.id=ce18eaaba3a6477684fff1fd98ec8b51) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:48:06.312787 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c524ed1-7726-4c03-8407-e84487691a56 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:06.314482 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c524ed1-7726-4c03-8407-e84487691a56 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:06.323448 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 351/678] 10.222.0.46 () {56 vars in 1374 bytes} [Thu Jun 23 15:48:06 2022] GET /identity/v3/role_assignments?user.id=ce18eaaba3a6477684fff1fd98ec8b51&scope.project.id=28ecdf5b474c44098c722f677d2a0b77&role.id=bfa8ecadc3d5483e821420aa0f0fb41e => generated 608 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:48:06.324646 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c524ed1-7726-4c03-8407-e84487691a56 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:06.370398 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:06.400980 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 328/679] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jun 23 15:48:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 15:48:06.401734 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b7b584-d9cd-46be-b188-ecaaaf8e2625 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:06.480106 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c524ed1-7726-4c03-8407-e84487691a56 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:06.543665 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1c27cc-8908-44c0-aca6-c970db58965f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:06.544327 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1c27cc-8908-44c0-aca6-c970db58965f None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:06.544615 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1c27cc-8908-44c0-aca6-c970db58965f None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:06.544888 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1c27cc-8908-44c0-aca6-c970db58965f None admin] PATH_INFO: `/v3/domains` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:06.545626 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1c27cc-8908-44c0-aca6-c970db58965f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:48:06.545928 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1c27cc-8908-44c0-aca6-c970db58965f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:06.547541 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1c27cc-8908-44c0-aca6-c970db58965f None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:06.548881 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b7b584-d9cd-46be-b188-ecaaaf8e2625 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:06.564044 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 352/680] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Jun 23 15:48:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:48:06.565688 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1c27cc-8908-44c0-aca6-c970db58965f None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:06.570750 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:06.603101 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65edcfee-f72e-49c8-a2aa-2cce19f5f712 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:06.604620 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65edcfee-f72e-49c8-a2aa-2cce19f5f712 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:06.605133 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65edcfee-f72e-49c8-a2aa-2cce19f5f712 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:06.605785 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65edcfee-f72e-49c8-a2aa-2cce19f5f712 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:06.607296 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65edcfee-f72e-49c8-a2aa-2cce19f5f712 None admin] RBAC: Adding query filter params (role.id=bfa8ecadc3d5483e821420aa0f0fb41e, scope.project.id=28ecdf5b474c44098c722f677d2a0b77, user.id=83970271286e451e9008bbf07cd3fa86) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:48:06.608103 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65edcfee-f72e-49c8-a2aa-2cce19f5f712 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:06.611020 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65edcfee-f72e-49c8-a2aa-2cce19f5f712 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:06.622865 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 329/681] 10.222.0.46 () {56 vars in 1374 bytes} [Thu Jun 23 15:48:06 2022] GET /identity/v3/role_assignments?user.id=83970271286e451e9008bbf07cd3fa86&scope.project.id=28ecdf5b474c44098c722f677d2a0b77&role.id=bfa8ecadc3d5483e821420aa0f0fb41e => generated 608 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:48:06.623863 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65edcfee-f72e-49c8-a2aa-2cce19f5f712 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:06.685786 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:06.714971 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1c27cc-8908-44c0-aca6-c970db58965f None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:06.759448 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebfb425-0c49-4752-861e-497f1f73bff4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:06.760628 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebfb425-0c49-4752-861e-497f1f73bff4 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:06.761112 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebfb425-0c49-4752-861e-497f1f73bff4 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:06.761617 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebfb425-0c49-4752-861e-497f1f73bff4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:06.770138 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebfb425-0c49-4752-861e-497f1f73bff4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:06.772154 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebfb425-0c49-4752-861e-497f1f73bff4 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:06.779446 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65edcfee-f72e-49c8-a2aa-2cce19f5f712 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:06.790501 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-1ebfb425-0c49-4752-861e-497f1f73bff4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 15:48:06.790501 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:48:06.790501 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:48:06.790501 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:48:06.790501 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:48:06.790501 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:48:06.790501 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:48:06.790501 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:48:06.790501 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:48:06.790501 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:48:06.790501 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:48:06.790501 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:48:06.790501 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 15:48:06.790501 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 15:48:06.790501 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 15:48:06.791503 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 15:48:06.791503 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:48:06.791503 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:48:06.791503 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:48:06.791503 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:48:06.791503 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:48:06.791503 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:48:06.791503 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:48:06.791503 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Jun 23 15:48:06.791503 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:48:06.791503 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:48:06.791503 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:48:06.791503 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:48:06.791503 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:48:06.791503 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:48:06.791503 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:48:06.791503 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:48:06.792566 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 15:48:06.792566 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 15:48:06.792566 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 15:48:06.792566 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 15:48:06.792566 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 15:48:06.792566 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 15:48:06.792566 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 15:48:06.792566 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Jun 23 15:48:06.795189 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 353/682] 10.222.0.46 () {56 vars in 1111 bytes} [Thu Jun 23 15:48:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:48:06.796348 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebfb425-0c49-4752-861e-497f1f73bff4 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:06.846118 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219c7f61-9d5a-48aa-9530-f7390b6f097b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:06.847345 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219c7f61-9d5a-48aa-9530-f7390b6f097b None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:06.847782 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219c7f61-9d5a-48aa-9530-f7390b6f097b None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:06.848205 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219c7f61-9d5a-48aa-9530-f7390b6f097b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:06.868368 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219c7f61-9d5a-48aa-9530-f7390b6f097b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 15:48:06.878229 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:06.878956 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-219c7f61-9d5a-48aa-9530-f7390b6f097b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 15:48:06.878956 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:48:06.878956 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:48:06.878956 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:48:06.878956 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:48:06.878956 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:48:06.878956 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:48:06.878956 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:48:06.878956 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:48:06.878956 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:48:06.878956 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:48:06.878956 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:48:06.878956 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 15:48:06.878956 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 15:48:06.878956 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 15:48:06.878956 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 15:48:06.880040 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 15:48:06.880040 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 15:48:06.880040 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 15:48:06.880040 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 15:48:06.880040 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:48:06.880040 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:48:06.880040 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:48:06.880040 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:48:06.880040 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:48:06.880040 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:48:06.880040 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:48:06.880040 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Jun 23 15:48:06.880040 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:48:06.880040 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:48:06.880040 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:48:06.880040 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:48:06.880925 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:48:06.880925 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:48:06.880925 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:48:06.880925 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:48:06.880925 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 15:48:06.880925 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 15:48:06.880925 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 15:48:06.880925 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 15:48:06.880925 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 15:48:06.880925 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 15:48:06.880925 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 15:48:06.880925 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Jun 23 15:48:06.880925 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 330/683] 10.222.0.46 () {56 vars in 1096 bytes} [Thu Jun 23 15:48:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:48:06.882166 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219c7f61-9d5a-48aa-9530-f7390b6f097b None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:06.939766 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebfb425-0c49-4752-861e-497f1f73bff4 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:06.990911 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80b77c2-7652-4d69-9524-61bd69f1a086 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:06.991730 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80b77c2-7652-4d69-9524-61bd69f1a086 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:06.992015 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80b77c2-7652-4d69-9524-61bd69f1a086 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:06.992292 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80b77c2-7652-4d69-9524-61bd69f1a086 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:07.000963 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80b77c2-7652-4d69-9524-61bd69f1a086 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:07.002436 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80b77c2-7652-4d69-9524-61bd69f1a086 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:07.019816 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-e80b77c2-7652-4d69-9524-61bd69f1a086 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 23 15:48:07.019816 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:48:07.019816 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:48:07.019816 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:48:07.019816 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:48:07.019816 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:48:07.019816 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:48:07.019816 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:48:07.019816 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:48:07.019816 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:48:07.019816 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:48:07.019816 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:48:07.019816 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 15:48:07.019816 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 15:48:07.019816 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 15:48:07.019816 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 15:48:07.020821 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:48:07.020821 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:48:07.020821 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 15:48:07.020821 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 15:48:07.020821 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 15:48:07.020821 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 15:48:07.020821 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:48:07.020821 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:48:07.020821 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:48:07.020821 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:48:07.020821 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:48:07.020821 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Jun 23 15:48:07.020821 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:48:07.020821 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:48:07.020821 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:48:07.020821 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:48:07.020821 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:48:07.020821 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:48:07.022327 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:48:07.022327 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:48:07.022327 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 15:48:07.022327 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 15:48:07.022327 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 15:48:07.022327 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 15:48:07.022327 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 15:48:07.022327 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 15:48:07.022327 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 23 15:48:07.022327 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Jun 23 15:48:07.022908 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 354/684] 10.222.0.46 () {56 vars in 1096 bytes} [Thu Jun 23 15:48:06 2022] GET /identity/v3/users/swift => generated 83 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:48:07.023532 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80b77c2-7652-4d69-9524-61bd69f1a086 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:07.046318 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219c7f61-9d5a-48aa-9530-f7390b6f097b None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:07.050051 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:07.096066 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0528ea4-3d59-4dd0-8d85-a642ee3972cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:07.096965 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0528ea4-3d59-4dd0-8d85-a642ee3972cf None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:07.097319 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0528ea4-3d59-4dd0-8d85-a642ee3972cf None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:07.097803 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0528ea4-3d59-4dd0-8d85-a642ee3972cf None admin] PATH_INFO: `/v3/projects` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:07.100501 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0528ea4-3d59-4dd0-8d85-a642ee3972cf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:48:07.100944 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0528ea4-3d59-4dd0-8d85-a642ee3972cf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:07.103594 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0528ea4-3d59-4dd0-8d85-a642ee3972cf None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:07.127365 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 331/685] 10.222.0.46 () {56 vars in 1148 bytes} [Thu Jun 23 15:48:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:48:07.128503 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0528ea4-3d59-4dd0-8d85-a642ee3972cf None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:07.139088 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80b77c2-7652-4d69-9524-61bd69f1a086 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:07.170036 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:07.186244 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2ae8cd-d38b-45b2-baf4-3490e81c507e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:07.187287 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2ae8cd-d38b-45b2-baf4-3490e81c507e None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:07.187753 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2ae8cd-d38b-45b2-baf4-3490e81c507e None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:07.188207 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2ae8cd-d38b-45b2-baf4-3490e81c507e None admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:07.189172 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2ae8cd-d38b-45b2-baf4-3490e81c507e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:48:07.189661 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2ae8cd-d38b-45b2-baf4-3490e81c507e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:07.191330 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2ae8cd-d38b-45b2-baf4-3490e81c507e None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:07.205694 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 355/686] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Jun 23 15:48:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:48:07.206986 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2ae8cd-d38b-45b2-baf4-3490e81c507e None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:07.304091 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0528ea4-3d59-4dd0-8d85-a642ee3972cf None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:07.340312 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2ae8cd-d38b-45b2-baf4-3490e81c507e None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:07.352581 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549a9329-0ef6-4e90-9f1f-a497cebf07f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:07.353204 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549a9329-0ef6-4e90-9f1f-a497cebf07f1 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:07.353519 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549a9329-0ef6-4e90-9f1f-a497cebf07f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:07.353777 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549a9329-0ef6-4e90-9f1f-a497cebf07f1 None admin] PATH_INFO: `/v3/users` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:07.355102 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549a9329-0ef6-4e90-9f1f-a497cebf07f1 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:48:07.355420 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549a9329-0ef6-4e90-9f1f-a497cebf07f1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:07.357191 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549a9329-0ef6-4e90-9f1f-a497cebf07f1 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:07.389394 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 332/687] 10.222.0.46 () {56 vars in 1135 bytes} [Thu Jun 23 15:48:07 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:48:07.390587 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549a9329-0ef6-4e90-9f1f-a497cebf07f1 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:07.401230 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae0323f-3fe2-4794-b552-0c6df2356a88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:07.402315 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae0323f-3fe2-4794-b552-0c6df2356a88 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:07.402811 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae0323f-3fe2-4794-b552-0c6df2356a88 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:07.403103 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae0323f-3fe2-4794-b552-0c6df2356a88 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:07.404272 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae0323f-3fe2-4794-b552-0c6df2356a88 None admin] RBAC: Adding query filter params (role.id=bfa8ecadc3d5483e821420aa0f0fb41e, scope.project.id=28ecdf5b474c44098c722f677d2a0b77, user.id=51784064951a4619ae09b9167fb0fd0d) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:48:07.404858 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae0323f-3fe2-4794-b552-0c6df2356a88 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:07.407063 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae0323f-3fe2-4794-b552-0c6df2356a88 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:07.418009 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 356/688] 10.222.0.46 () {56 vars in 1374 bytes} [Thu Jun 23 15:48:07 2022] GET /identity/v3/role_assignments?user.id=51784064951a4619ae09b9167fb0fd0d&scope.project.id=28ecdf5b474c44098c722f677d2a0b77&role.id=bfa8ecadc3d5483e821420aa0f0fb41e => generated 608 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:48:07.419267 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae0323f-3fe2-4794-b552-0c6df2356a88 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:07.434729 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:07.520398 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:07.606169 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549a9329-0ef6-4e90-9f1f-a497cebf07f1 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:07.646727 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae0323f-3fe2-4794-b552-0c6df2356a88 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:07.657836 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31298a99-dee1-4ed7-a2f8-b8b19a9d2292 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:07.658960 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31298a99-dee1-4ed7-a2f8-b8b19a9d2292 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:07.661639 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31298a99-dee1-4ed7-a2f8-b8b19a9d2292 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:07.662107 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31298a99-dee1-4ed7-a2f8-b8b19a9d2292 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:07.680070 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31298a99-dee1-4ed7-a2f8-b8b19a9d2292 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:07.682477 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31298a99-dee1-4ed7-a2f8-b8b19a9d2292 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:07.711713 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787b213f-9451-4789-a0da-cd28157a7cda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:07.712396 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787b213f-9451-4789-a0da-cd28157a7cda None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:07.712920 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787b213f-9451-4789-a0da-cd28157a7cda None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:07.713380 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787b213f-9451-4789-a0da-cd28157a7cda None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:07.716668 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-31298a99-dee1-4ed7-a2f8-b8b19a9d2292 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 15:48:07.716668 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:48:07.716668 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:48:07.716668 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:48:07.716668 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:48:07.716668 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:48:07.716668 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:48:07.716668 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:48:07.716668 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:48:07.716668 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:48:07.716668 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:48:07.716668 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:48:07.716668 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 15:48:07.716668 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 15:48:07.716668 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 15:48:07.718318 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 15:48:07.718318 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:48:07.718318 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:48:07.718318 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:48:07.718318 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:48:07.718318 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:48:07.718318 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:48:07.718318 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:48:07.718318 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Jun 23 15:48:07.718318 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:48:07.718318 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:48:07.718318 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:48:07.718318 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:48:07.718318 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:48:07.718318 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:48:07.718318 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:48:07.718318 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:48:07.719740 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 15:48:07.719740 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 15:48:07.719740 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 15:48:07.719740 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Jun 23 15:48:07.723321 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 333/689] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Jun 23 15:48:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 333 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:48:07.730129 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31298a99-dee1-4ed7-a2f8-b8b19a9d2292 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:07.819689 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787b213f-9451-4789-a0da-cd28157a7cda None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:07.821782 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787b213f-9451-4789-a0da-cd28157a7cda None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:07.835614 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:07.864292 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31298a99-dee1-4ed7-a2f8-b8b19a9d2292 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:07.885342 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:07.903009 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216e68e7-a2ba-4e0e-9c43-2bbdb6079a81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:07.903958 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216e68e7-a2ba-4e0e-9c43-2bbdb6079a81 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:07.904390 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216e68e7-a2ba-4e0e-9c43-2bbdb6079a81 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:07.904827 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216e68e7-a2ba-4e0e-9c43-2bbdb6079a81 None admin] PATH_INFO: `/v3/domains` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:07.905705 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216e68e7-a2ba-4e0e-9c43-2bbdb6079a81 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:48:07.906161 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216e68e7-a2ba-4e0e-9c43-2bbdb6079a81 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:07.907936 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216e68e7-a2ba-4e0e-9c43-2bbdb6079a81 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:07.922000 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 334/690] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Jun 23 15:48:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:48:07.926517 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216e68e7-a2ba-4e0e-9c43-2bbdb6079a81 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:07.977215 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 357/691] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Jun 23 15:48:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 559 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 15:48:07.983986 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787b213f-9451-4789-a0da-cd28157a7cda None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:08.155524 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216e68e7-a2ba-4e0e-9c43-2bbdb6079a81 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:08.196558 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91926ceb-1672-4e01-896a-cde7e8dc7bb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:08.197670 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91926ceb-1672-4e01-896a-cde7e8dc7bb0 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:08.198137 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91926ceb-1672-4e01-896a-cde7e8dc7bb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:08.198615 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91926ceb-1672-4e01-896a-cde7e8dc7bb0 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:08.201252 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:08.210732 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91926ceb-1672-4e01-896a-cde7e8dc7bb0 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:08.212509 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91926ceb-1672-4e01-896a-cde7e8dc7bb0 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:08.229981 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787b213f-9451-4789-a0da-cd28157a7cda None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:08.230874 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-91926ceb-1672-4e01-896a-cde7e8dc7bb0 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 23 15:48:08.230874 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:48:08.230874 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:48:08.230874 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:48:08.230874 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:48:08.230874 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:48:08.230874 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:48:08.230874 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:48:08.230874 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:48:08.230874 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:48:08.230874 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:48:08.230874 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:48:08.230874 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 15:48:08.230874 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 15:48:08.230874 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 15:48:08.230874 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 15:48:08.231871 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:48:08.231871 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:48:08.231871 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 15:48:08.231871 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 15:48:08.231871 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 15:48:08.231871 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 15:48:08.231871 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:48:08.231871 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:48:08.231871 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:48:08.231871 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:48:08.231871 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:48:08.231871 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Jun 23 15:48:08.231871 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:48:08.231871 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:48:08.231871 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:48:08.231871 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:48:08.231871 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:48:08.231871 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:48:08.232813 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:48:08.232813 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:48:08.232813 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 15:48:08.232813 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 15:48:08.232813 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 15:48:08.232813 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 15:48:08.232813 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 15:48:08.232813 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 15:48:08.232813 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 23 15:48:08.232813 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Jun 23 15:48:08.235857 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 335/692] 10.222.0.46 () {56 vars in 1093 bytes} [Thu Jun 23 15:48:07 2022] GET /identity/v3/users/nova => generated 82 bytes in 310 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 15:48:08.244799 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91926ceb-1672-4e01-896a-cde7e8dc7bb0 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:08.309940 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcc9c4e-46e6-48ce-aca5-98651980e1af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:08.311587 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcc9c4e-46e6-48ce-aca5-98651980e1af None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:08.312263 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcc9c4e-46e6-48ce-aca5-98651980e1af None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:08.312779 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcc9c4e-46e6-48ce-aca5-98651980e1af None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:08.331748 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcc9c4e-46e6-48ce-aca5-98651980e1af None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:08.333304 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcc9c4e-46e6-48ce-aca5-98651980e1af None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:08.379948 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91926ceb-1672-4e01-896a-cde7e8dc7bb0 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:08.380880 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-adcc9c4e-46e6-48ce-aca5-98651980e1af None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 15:48:08.380880 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:48:08.380880 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:48:08.380880 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:48:08.380880 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:48:08.380880 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:48:08.380880 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:48:08.380880 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:48:08.380880 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:48:08.380880 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:48:08.380880 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:48:08.380880 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:48:08.380880 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 15:48:08.380880 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 15:48:08.380880 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 15:48:08.381791 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 15:48:08.381791 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:48:08.381791 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:48:08.381791 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:48:08.381791 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:48:08.381791 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:48:08.381791 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:48:08.381791 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:48:08.381791 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Jun 23 15:48:08.381791 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:48:08.381791 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:48:08.381791 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:48:08.381791 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:48:08.381791 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:48:08.381791 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:48:08.381791 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:48:08.381791 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:48:08.382732 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 15:48:08.382732 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 15:48:08.382732 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 15:48:08.382732 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Jun 23 15:48:08.382732 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 358/693] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Jun 23 15:48:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 399 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:48:08.387829 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcc9c4e-46e6-48ce-aca5-98651980e1af None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:08.393493 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:08.430644 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67587ae-b84e-48bd-9031-646d232d75da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:08.431761 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67587ae-b84e-48bd-9031-646d232d75da None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:08.432248 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67587ae-b84e-48bd-9031-646d232d75da None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:08.432716 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67587ae-b84e-48bd-9031-646d232d75da None admin] PATH_INFO: `/v3/users` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:08.433752 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67587ae-b84e-48bd-9031-646d232d75da None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:48:08.434274 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67587ae-b84e-48bd-9031-646d232d75da None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:08.435968 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67587ae-b84e-48bd-9031-646d232d75da None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:08.456878 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 336/694] 10.222.0.46 () {56 vars in 1133 bytes} [Thu Jun 23 15:48:08 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 15:48:08.467730 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67587ae-b84e-48bd-9031-646d232d75da None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:08.538987 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcc9c4e-46e6-48ce-aca5-98651980e1af None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:08.589287 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b691c14f-32d5-4414-84c2-2fa36b627989 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:08.589931 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b691c14f-32d5-4414-84c2-2fa36b627989 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:08.590806 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b691c14f-32d5-4414-84c2-2fa36b627989 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:08.591094 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b691c14f-32d5-4414-84c2-2fa36b627989 None admin] PATH_INFO: `/v3/domains` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:08.592181 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b691c14f-32d5-4414-84c2-2fa36b627989 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:48:08.592488 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b691c14f-32d5-4414-84c2-2fa36b627989 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:08.593870 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b691c14f-32d5-4414-84c2-2fa36b627989 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:08.608041 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:08.614162 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 359/695] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Jun 23 15:48:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:48:08.623835 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b691c14f-32d5-4414-84c2-2fa36b627989 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:08.689708 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67587ae-b84e-48bd-9031-646d232d75da None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:08.729893 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:08.758041 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844c6a42-b0d9-4e8b-9474-de3910d2e47e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:08.759380 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844c6a42-b0d9-4e8b-9474-de3910d2e47e None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:08.759848 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844c6a42-b0d9-4e8b-9474-de3910d2e47e None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:08.760330 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844c6a42-b0d9-4e8b-9474-de3910d2e47e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:08.802837 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b691c14f-32d5-4414-84c2-2fa36b627989 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:08.853936 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc89433-137b-44a8-a83f-c76ec26977c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:08.855357 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc89433-137b-44a8-a83f-c76ec26977c0 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:08.855828 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc89433-137b-44a8-a83f-c76ec26977c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:08.856308 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc89433-137b-44a8-a83f-c76ec26977c0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:08.869596 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc89433-137b-44a8-a83f-c76ec26977c0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:08.872305 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc89433-137b-44a8-a83f-c76ec26977c0 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:08.885760 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844c6a42-b0d9-4e8b-9474-de3910d2e47e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:08.887548 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844c6a42-b0d9-4e8b-9474-de3910d2e47e None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:08.899122 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-4fc89433-137b-44a8-a83f-c76ec26977c0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 15:48:08.899122 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:48:08.899122 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:48:08.899122 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:48:08.899122 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:48:08.899122 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:48:08.899122 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:48:08.899122 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:48:08.899122 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:48:08.899122 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:48:08.899122 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:48:08.899122 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:48:08.899122 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 15:48:08.899122 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 15:48:08.899122 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 15:48:08.900291 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 15:48:08.900291 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:48:08.900291 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:48:08.900291 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:48:08.900291 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:48:08.900291 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:48:08.900291 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:48:08.900291 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:48:08.900291 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Jun 23 15:48:08.900291 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:48:08.900291 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:48:08.900291 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:48:08.900291 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:48:08.900291 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:48:08.900291 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:48:08.900291 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:48:08.900291 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:48:08.901185 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 15:48:08.901185 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 15:48:08.901185 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 15:48:08.901185 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 15:48:08.901185 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 15:48:08.901185 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 15:48:08.901185 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 15:48:08.901185 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Jun 23 15:48:08.905296 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 360/696] 10.222.0.46 () {56 vars in 1111 bytes} [Thu Jun 23 15:48:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:48:08.906773 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ded71ec-496e-4909-a577-68799dc8e240 None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:08.906972 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ded71ec-496e-4909-a577-68799dc8e240 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:08.907251 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ded71ec-496e-4909-a577-68799dc8e240 None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:08.908780 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 361/697] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jun 23 15:48:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 15:48:08.909524 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ded71ec-496e-4909-a577-68799dc8e240 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:08.951545 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:09.010791 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 337/698] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Jun 23 15:48:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 539 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 15:48:09.012489 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f440ce8-9cb7-4af6-813e-5a2301d53b33 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:09.012959 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f440ce8-9cb7-4af6-813e-5a2301d53b33 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:09.013412 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f440ce8-9cb7-4af6-813e-5a2301d53b33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:09.031128 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-3f440ce8-9cb7-4af6-813e-5a2301d53b33 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:48:09.032325 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-3f440ce8-9cb7-4af6-813e-5a2301d53b33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:48:09.046118 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ded71ec-496e-4909-a577-68799dc8e240 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:09.073087 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:09.088126 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9344664e-d8cd-4949-9df3-8f904cce0e61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:09.088794 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9344664e-d8cd-4949-9df3-8f904cce0e61 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:09.089076 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9344664e-d8cd-4949-9df3-8f904cce0e61 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:09.089465 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9344664e-d8cd-4949-9df3-8f904cce0e61 None admin] PATH_INFO: `/v3/projects` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:09.090253 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9344664e-d8cd-4949-9df3-8f904cce0e61 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:48:09.090567 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9344664e-d8cd-4949-9df3-8f904cce0e61 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:09.092180 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9344664e-d8cd-4949-9df3-8f904cce0e61 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:09.104186 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 338/699] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jun 23 15:48:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 15:48:09.104835 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f440ce8-9cb7-4af6-813e-5a2301d53b33 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:09.110735 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 362/700] 10.222.0.46 () {56 vars in 1148 bytes} [Thu Jun 23 15:48:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:48:09.111097 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce396a74-e601-49d2-a946-7c264244e6ec None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:09.111251 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce396a74-e601-49d2-a946-7c264244e6ec None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:09.111504 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce396a74-e601-49d2-a946-7c264244e6ec None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:09.113220 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 363/701] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jun 23 15:48:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 15:48:09.113847 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce396a74-e601-49d2-a946-7c264244e6ec None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:09.198223 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:09.218992 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f440ce8-9cb7-4af6-813e-5a2301d53b33 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:09.239446 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce396a74-e601-49d2-a946-7c264244e6ec None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:09.256926 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6e9c54-bd18-4103-a4fc-049f19f0d845 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:09.257958 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6e9c54-bd18-4103-a4fc-049f19f0d845 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:09.258248 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6e9c54-bd18-4103-a4fc-049f19f0d845 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:09.258513 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6e9c54-bd18-4103-a4fc-049f19f0d845 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:09.269886 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6e9c54-bd18-4103-a4fc-049f19f0d845 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:09.271519 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6e9c54-bd18-4103-a4fc-049f19f0d845 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:09.277108 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2f4091-b26c-4cda-b8ac-8d8dbd63106b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:09.277770 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2f4091-b26c-4cda-b8ac-8d8dbd63106b None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:09.278370 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2f4091-b26c-4cda-b8ac-8d8dbd63106b None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:09.278370 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2f4091-b26c-4cda-b8ac-8d8dbd63106b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:09.279184 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2f4091-b26c-4cda-b8ac-8d8dbd63106b None admin] RBAC: Adding query filter params (role.id=bfa8ecadc3d5483e821420aa0f0fb41e, scope.project.id=28ecdf5b474c44098c722f677d2a0b77, user.id=242bf02d14724fe4837fcce8730377f9) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:48:09.279509 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2f4091-b26c-4cda-b8ac-8d8dbd63106b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:09.280919 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2f4091-b26c-4cda-b8ac-8d8dbd63106b None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:09.287096 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 364/702] 10.222.0.46 () {56 vars in 1374 bytes} [Thu Jun 23 15:48:09 2022] GET /identity/v3/role_assignments?user.id=242bf02d14724fe4837fcce8730377f9&scope.project.id=28ecdf5b474c44098c722f677d2a0b77&role.id=bfa8ecadc3d5483e821420aa0f0fb41e => generated 608 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:48:09.289420 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-1f6e9c54-bd18-4103-a4fc-049f19f0d845 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 15:48:09.289420 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:48:09.289420 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:48:09.289420 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:48:09.289420 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:48:09.289420 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:48:09.289420 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:48:09.289420 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:48:09.289420 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:48:09.289420 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:48:09.289420 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:48:09.289420 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:48:09.289420 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 15:48:09.289420 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 15:48:09.289420 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 15:48:09.290285 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 15:48:09.290285 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:48:09.290285 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:48:09.290285 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:48:09.290285 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:48:09.290285 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:48:09.290285 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:48:09.290285 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:48:09.290285 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Jun 23 15:48:09.290285 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:48:09.290285 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:48:09.290285 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:48:09.290285 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:48:09.290285 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:48:09.290285 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:48:09.290285 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:48:09.290285 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:48:09.291649 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 15:48:09.291649 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 15:48:09.291649 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 15:48:09.291649 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Jun 23 15:48:09.295250 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 339/703] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Jun 23 15:48:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:48:09.302893 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6e9c54-bd18-4103-a4fc-049f19f0d845 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:09.303129 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d33b7f-86b1-4a9b-9415-0695e54fb158 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:09.303129 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d33b7f-86b1-4a9b-9415-0695e54fb158 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:09.303129 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d33b7f-86b1-4a9b-9415-0695e54fb158 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:09.320972 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-f2d33b7f-86b1-4a9b-9415-0695e54fb158 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:48:09.322217 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-f2d33b7f-86b1-4a9b-9415-0695e54fb158 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:48:09.372991 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:09.404551 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 365/704] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jun 23 15:48:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 15:48:09.405719 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d33b7f-86b1-4a9b-9415-0695e54fb158 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:09.436917 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6e9c54-bd18-4103-a4fc-049f19f0d845 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:09.486552 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4179bfa-3dd4-40f0-87b2-3ef7126fff78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:09.487359 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4179bfa-3dd4-40f0-87b2-3ef7126fff78 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:09.487640 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4179bfa-3dd4-40f0-87b2-3ef7126fff78 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:09.487914 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4179bfa-3dd4-40f0-87b2-3ef7126fff78 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:09.488782 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4179bfa-3dd4-40f0-87b2-3ef7126fff78 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:09.491630 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4179bfa-3dd4-40f0-87b2-3ef7126fff78 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:09.497758 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:09.510124 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-f4179bfa-3dd4-40f0-87b2-3ef7126fff78 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 23 15:48:09.510124 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:48:09.510124 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 15:48:09.510124 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 15:48:09.510124 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 15:48:09.510124 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 15:48:09.510124 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 15:48:09.510124 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 15:48:09.510124 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 23 15:48:09.510124 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Jun 23 15:48:09.510124 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 15:48:09.510124 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Jun 23 15:48:09.510124 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:48:09.510124 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:48:09.510124 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:48:09.510124 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:48:09.511374 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:48:09.511374 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:48:09.511374 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:48:09.511374 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:48:09.511374 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:48:09.511374 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:48:09.511374 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:48:09.511374 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 15:48:09.511374 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 15:48:09.511374 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 15:48:09.511374 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 15:48:09.511374 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:48:09.511374 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:48:09.511374 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:48:09.511374 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:48:09.511374 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:48:09.512269 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:48:09.512269 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:48:09.512269 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Jun 23 15:48:09.512269 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:48:09.512269 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:48:09.512269 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:48:09.512269 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:48:09.512269 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:48:09.512269 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:48:09.512269 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:48:09.512269 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:48:09.512269 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 15:48:09.512269 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 15:48:09.512269 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 23 15:48:09.512269 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Jun 23 15:48:09.515656 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 340/705] 10.222.0.46 () {56 vars in 1129 bytes} [Thu Jun 23 15:48:09 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:48:09.516755 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4179bfa-3dd4-40f0-87b2-3ef7126fff78 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:09.534924 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d33b7f-86b1-4a9b-9415-0695e54fb158 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:09.603880 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c7be41-9708-4290-99d2-bf5042dfb1bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:09.604816 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c7be41-9708-4290-99d2-bf5042dfb1bc None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:09.605102 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c7be41-9708-4290-99d2-bf5042dfb1bc None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:09.605382 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c7be41-9708-4290-99d2-bf5042dfb1bc None admin] PATH_INFO: `/v3/domains` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:09.606123 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c7be41-9708-4290-99d2-bf5042dfb1bc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:48:09.606432 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c7be41-9708-4290-99d2-bf5042dfb1bc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:09.607895 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c7be41-9708-4290-99d2-bf5042dfb1bc None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:09.622432 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 366/706] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Jun 23 15:48:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:48:09.623178 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c7be41-9708-4290-99d2-bf5042dfb1bc None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:09.652294 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4179bfa-3dd4-40f0-87b2-3ef7126fff78 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:09.666415 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:09.714972 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824acfb1-cab5-403a-bdea-be90096f3239 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:09.716444 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824acfb1-cab5-403a-bdea-be90096f3239 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:09.716957 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824acfb1-cab5-403a-bdea-be90096f3239 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:09.717473 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824acfb1-cab5-403a-bdea-be90096f3239 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:09.737599 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824acfb1-cab5-403a-bdea-be90096f3239 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:09.740709 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824acfb1-cab5-403a-bdea-be90096f3239 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:09.744857 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c7be41-9708-4290-99d2-bf5042dfb1bc None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:09.777153 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-824acfb1-cab5-403a-bdea-be90096f3239 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 15:48:09.777153 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:48:09.777153 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:48:09.777153 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:48:09.777153 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:48:09.777153 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:48:09.777153 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:48:09.777153 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:48:09.777153 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:48:09.777153 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:48:09.777153 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:48:09.777153 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:48:09.777153 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 15:48:09.777153 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 15:48:09.777153 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 15:48:09.778502 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 15:48:09.778502 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:48:09.778502 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:48:09.778502 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:48:09.778502 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:48:09.778502 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:48:09.778502 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:48:09.778502 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:48:09.778502 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Jun 23 15:48:09.778502 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:48:09.778502 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:48:09.778502 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:48:09.778502 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:48:09.778502 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:48:09.778502 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:48:09.778502 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:48:09.778502 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:48:09.779527 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 15:48:09.779527 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 15:48:09.779527 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 15:48:09.779527 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Jun 23 15:48:09.779925 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 341/707] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Jun 23 15:48:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:48:09.781946 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824acfb1-cab5-403a-bdea-be90096f3239 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:09.789583 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d128180a-4a83-486a-9827-e8a64bf7302e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:09.790408 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d128180a-4a83-486a-9827-e8a64bf7302e None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:09.790819 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d128180a-4a83-486a-9827-e8a64bf7302e None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:09.791166 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d128180a-4a83-486a-9827-e8a64bf7302e None admin] PATH_INFO: `/v3/services` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:09.791955 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d128180a-4a83-486a-9827-e8a64bf7302e None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:48:09.792331 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d128180a-4a83-486a-9827-e8a64bf7302e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:09.793795 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d128180a-4a83-486a-9827-e8a64bf7302e None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:09.799126 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 367/708] 10.222.0.46 () {56 vars in 1124 bytes} [Thu Jun 23 15:48:09 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:48:09.800013 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d128180a-4a83-486a-9827-e8a64bf7302e None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:09.850859 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:09.905276 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824acfb1-cab5-403a-bdea-be90096f3239 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:09.908635 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d128180a-4a83-486a-9827-e8a64bf7302e None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:09.911078 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:09.942873 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cd1aa8-7bdf-49c5-bd3c-9f0775d72114 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:09.943903 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6f93e3-56a3-45ec-a517-13eadb3991e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:09.944341 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cd1aa8-7bdf-49c5-bd3c-9f0775d72114 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:09.944569 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6f93e3-56a3-45ec-a517-13eadb3991e3 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:09.944853 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6f93e3-56a3-45ec-a517-13eadb3991e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:09.945128 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6f93e3-56a3-45ec-a517-13eadb3991e3 None admin] PATH_INFO: `/v3/domains` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:09.945496 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cd1aa8-7bdf-49c5-bd3c-9f0775d72114 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:09.945852 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6f93e3-56a3-45ec-a517-13eadb3991e3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:48:09.946157 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6f93e3-56a3-45ec-a517-13eadb3991e3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:09.946373 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cd1aa8-7bdf-49c5-bd3c-9f0775d72114 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:09.947586 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6f93e3-56a3-45ec-a517-13eadb3991e3 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:09.954292 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24cd1aa8-7bdf-49c5-bd3c-9f0775d72114 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:09.956054 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24cd1aa8-7bdf-49c5-bd3c-9f0775d72114 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:09.962818 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 368/709] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Jun 23 15:48:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:48:09.962818 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6f93e3-56a3-45ec-a517-13eadb3991e3 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:09.977283 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-24cd1aa8-7bdf-49c5-bd3c-9f0775d72114 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 15:48:09.977283 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:48:09.977283 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:48:09.977283 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:48:09.977283 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:48:09.977283 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:48:09.977283 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:48:09.977283 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:48:09.977283 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:48:09.977283 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:48:09.977283 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:48:09.977283 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:48:09.977283 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 15:48:09.977283 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 15:48:09.977283 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 15:48:09.978262 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 15:48:09.978262 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:48:09.978262 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:48:09.978262 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:48:09.978262 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:48:09.978262 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:48:09.978262 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:48:09.978262 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:48:09.978262 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Jun 23 15:48:09.978262 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:48:09.978262 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:48:09.978262 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:48:09.978262 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:48:09.978262 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:48:09.978262 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:48:09.978262 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:48:09.978262 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:48:09.979213 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 15:48:09.979213 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 15:48:09.979213 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 15:48:09.979213 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 15:48:09.979213 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 15:48:09.979213 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 15:48:09.979213 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 15:48:09.979213 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Jun 23 15:48:09.979213 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 342/710] 10.222.0.46 () {56 vars in 1111 bytes} [Thu Jun 23 15:48:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:48:09.979699 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cd1aa8-7bdf-49c5-bd3c-9f0775d72114 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:10.079672 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6f93e3-56a3-45ec-a517-13eadb3991e3 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:10.108708 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cd1aa8-7bdf-49c5-bd3c-9f0775d72114 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:10.120339 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131a9405-36e3-4dcb-81d1-638a9fd128f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:10.121459 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131a9405-36e3-4dcb-81d1-638a9fd128f6 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:10.121743 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131a9405-36e3-4dcb-81d1-638a9fd128f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:10.122021 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131a9405-36e3-4dcb-81d1-638a9fd128f6 None admin] PATH_INFO: `/v3/services` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:10.122719 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:10.124495 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131a9405-36e3-4dcb-81d1-638a9fd128f6 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:48:10.124810 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131a9405-36e3-4dcb-81d1-638a9fd128f6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:10.127367 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131a9405-36e3-4dcb-81d1-638a9fd128f6 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:10.133685 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 369/711] 10.222.0.46 () {56 vars in 1124 bytes} [Thu Jun 23 15:48:09 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:48:10.134493 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131a9405-36e3-4dcb-81d1-638a9fd128f6 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:10.154195 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94862865-02eb-4bbf-9e82-752398f4b328 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:10.154972 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94862865-02eb-4bbf-9e82-752398f4b328 None admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:10.155259 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94862865-02eb-4bbf-9e82-752398f4b328 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:10.155553 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94862865-02eb-4bbf-9e82-752398f4b328 None admin] PATH_INFO: `/v3/users` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:10.156605 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94862865-02eb-4bbf-9e82-752398f4b328 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:10.157960 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94862865-02eb-4bbf-9e82-752398f4b328 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:10.169824 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:10.190758 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 343/712] 10.222.0.46 () {60 vars in 1132 bytes} [Thu Jun 23 15:48:09 2022] POST /identity/v3/users => generated 254 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 15:48:10.192891 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ceb60c-572c-44be-aca1-ab508096e9ab None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:10.193555 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ceb60c-572c-44be-aca1-ab508096e9ab None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:10.194044 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ceb60c-572c-44be-aca1-ab508096e9ab None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:10.196837 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 344/713] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jun 23 15:48:10 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 15:48:10.205119 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1ee912-3c86-4e2a-be75-863118e36508 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:10.205461 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1ee912-3c86-4e2a-be75-863118e36508 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:10.205752 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1ee912-3c86-4e2a-be75-863118e36508 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:10.242925 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-5e1ee912-3c86-4e2a-be75-863118e36508 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:48:10.244188 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-5e1ee912-3c86-4e2a-be75-863118e36508 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:48:10.320339 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131a9405-36e3-4dcb-81d1-638a9fd128f6 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:10.360828 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 345/714] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jun 23 15:48:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 15:48:10.375208 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115796f9-e0ff-4042-9f7d-19aac9cf2695 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:10.375907 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115796f9-e0ff-4042-9f7d-19aac9cf2695 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:10.376193 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115796f9-e0ff-4042-9f7d-19aac9cf2695 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:10.376467 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115796f9-e0ff-4042-9f7d-19aac9cf2695 None admin] PATH_INFO: `/v3/projects` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:10.377349 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115796f9-e0ff-4042-9f7d-19aac9cf2695 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:48:10.377661 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115796f9-e0ff-4042-9f7d-19aac9cf2695 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:10.379215 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115796f9-e0ff-4042-9f7d-19aac9cf2695 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:10.392181 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1ee912-3c86-4e2a-be75-863118e36508 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:10.406466 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 370/715] 10.222.0.46 () {56 vars in 1148 bytes} [Thu Jun 23 15:48:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:48:10.413080 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115796f9-e0ff-4042-9f7d-19aac9cf2695 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:10.506416 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:10.512738 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1ee912-3c86-4e2a-be75-863118e36508 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:10.523520 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:10.542619 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115796f9-e0ff-4042-9f7d-19aac9cf2695 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:10.563429 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3590dd1b-df1b-4aba-971a-02996a5fda4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:10.564245 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3590dd1b-df1b-4aba-971a-02996a5fda4c None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:10.564526 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3590dd1b-df1b-4aba-971a-02996a5fda4c None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:10.564797 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3590dd1b-df1b-4aba-971a-02996a5fda4c None admin] PATH_INFO: `/v3/services/network` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:10.565453 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3590dd1b-df1b-4aba-971a-02996a5fda4c None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:10.567314 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3590dd1b-df1b-4aba-971a-02996a5fda4c None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:10.588701 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-3590dd1b-df1b-4aba-971a-02996a5fda4c None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 23 15:48:10.588701 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:48:10.588701 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 15:48:10.588701 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 15:48:10.588701 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 15:48:10.588701 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 15:48:10.588701 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 15:48:10.588701 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 15:48:10.588701 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 23 15:48:10.588701 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Jun 23 15:48:10.588701 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 15:48:10.588701 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Jun 23 15:48:10.588701 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:48:10.588701 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:48:10.588701 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:48:10.588701 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:48:10.588701 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:48:10.589731 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:48:10.589731 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:48:10.589731 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:48:10.589731 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:48:10.589731 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:48:10.589731 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:48:10.589731 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 15:48:10.589731 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 15:48:10.589731 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 15:48:10.589731 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 15:48:10.589731 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:48:10.589731 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:48:10.589731 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:48:10.589731 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:48:10.589731 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:48:10.589731 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:48:10.590568 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:48:10.590568 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Jun 23 15:48:10.590568 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:48:10.590568 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:48:10.590568 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:48:10.590568 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:48:10.590568 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:48:10.590568 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:48:10.590568 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:48:10.590568 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:48:10.590568 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 15:48:10.590568 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 15:48:10.590568 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 23 15:48:10.590568 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Jun 23 15:48:10.590568 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 346/716] 10.222.0.46 () {56 vars in 1111 bytes} [Thu Jun 23 15:48:10 2022] GET /identity/v3/services/network => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 15:48:10.593428 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0406fea1-5594-467a-a432-544d6c999e61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:10.594353 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0406fea1-5594-467a-a432-544d6c999e61 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:10.594879 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0406fea1-5594-467a-a432-544d6c999e61 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:10.595366 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0406fea1-5594-467a-a432-544d6c999e61 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:10.596420 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0406fea1-5594-467a-a432-544d6c999e61 None admin] RBAC: Adding query filter params (role.id=812d3c567c52469cb2b35709fbdc857c, scope.project.id=28ecdf5b474c44098c722f677d2a0b77, user.id=705115d19bab424d894bea8b1b48a7f0) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:48:10.596922 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3590dd1b-df1b-4aba-971a-02996a5fda4c None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:10.597149 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0406fea1-5594-467a-a432-544d6c999e61 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:10.598938 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0406fea1-5594-467a-a432-544d6c999e61 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:10.607159 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 371/717] 10.222.0.46 () {56 vars in 1374 bytes} [Thu Jun 23 15:48:10 2022] GET /identity/v3/role_assignments?user.id=705115d19bab424d894bea8b1b48a7f0&scope.project.id=28ecdf5b474c44098c722f677d2a0b77&role.id=812d3c567c52469cb2b35709fbdc857c => generated 259 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:48:10.733603 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3590dd1b-df1b-4aba-971a-02996a5fda4c None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:10.782247 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76aad30f-51d6-410e-b763-15cdadfae1c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:10.783088 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76aad30f-51d6-410e-b763-15cdadfae1c5 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:10.783259 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76aad30f-51d6-410e-b763-15cdadfae1c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:10.783514 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76aad30f-51d6-410e-b763-15cdadfae1c5 None admin] PATH_INFO: `/v3/services` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:10.784247 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76aad30f-51d6-410e-b763-15cdadfae1c5 None admin] RBAC: Adding query filter params (name=network) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:48:10.784551 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76aad30f-51d6-410e-b763-15cdadfae1c5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:10.785942 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76aad30f-51d6-410e-b763-15cdadfae1c5 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:10.794818 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 347/718] 10.222.0.46 () {56 vars in 1112 bytes} [Thu Jun 23 15:48:10 2022] GET /identity/v3/services?name=network => generated 124 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:48:10.800410 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0406fea1-5594-467a-a432-544d6c999e61 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:10.961739 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0406fea1-5594-467a-a432-544d6c999e61 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:10.998980 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f79876a-27b4-4084-8c3d-02460f211a5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:10.999577 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f79876a-27b4-4084-8c3d-02460f211a5f None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:10.999862 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f79876a-27b4-4084-8c3d-02460f211a5f None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:11.000138 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f79876a-27b4-4084-8c3d-02460f211a5f None admin] PATH_INFO: `/v3/services` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:11.000859 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f79876a-27b4-4084-8c3d-02460f211a5f None admin] RBAC: Adding query filter params (type=network) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:48:11.001171 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f79876a-27b4-4084-8c3d-02460f211a5f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:11.002554 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f79876a-27b4-4084-8c3d-02460f211a5f None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:11.008161 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 372/719] 10.222.0.46 () {56 vars in 1112 bytes} [Thu Jun 23 15:48:10 2022] GET /identity/v3/services?type=network => generated 124 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:48:11.808135 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d3f61c-7779-4e7d-b870-1ac384fbf236 None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:11.808648 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d3f61c-7779-4e7d-b870-1ac384fbf236 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:11.808726 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d3f61c-7779-4e7d-b870-1ac384fbf236 None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:11.810292 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 348/720] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jun 23 15:48:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 15:48:11.815979 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476ea5d6-df90-46e3-a9f8-354b5fee64d6 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:11.816421 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476ea5d6-df90-46e3-a9f8-354b5fee64d6 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:11.816605 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476ea5d6-df90-46e3-a9f8-354b5fee64d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:11.838676 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-476ea5d6-df90-46e3-a9f8-354b5fee64d6 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:48:11.839881 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-476ea5d6-df90-46e3-a9f8-354b5fee64d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:48:11.853389 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:11.934363 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 373/721] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jun 23 15:48:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 15:48:11.957847 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d3f61c-7779-4e7d-b870-1ac384fbf236 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:11.995214 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:12.103460 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d3f61c-7779-4e7d-b870-1ac384fbf236 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:12.149222 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b981bc-e80a-4f5e-87fa-0989fe15c8b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:12.150651 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b981bc-e80a-4f5e-87fa-0989fe15c8b0 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:12.151184 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b981bc-e80a-4f5e-87fa-0989fe15c8b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:12.151654 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b981bc-e80a-4f5e-87fa-0989fe15c8b0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:12.171527 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b981bc-e80a-4f5e-87fa-0989fe15c8b0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 15:48:12.179482 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-a0b981bc-e80a-4f5e-87fa-0989fe15c8b0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 15:48:12.179482 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:48:12.179482 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:48:12.179482 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:48:12.179482 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:48:12.179482 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:48:12.179482 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:48:12.179482 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:48:12.179482 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:48:12.179482 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:48:12.179482 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:48:12.179482 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:48:12.179482 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 15:48:12.179482 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 15:48:12.179482 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 15:48:12.179482 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 15:48:12.180385 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 15:48:12.180385 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 15:48:12.180385 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 15:48:12.180385 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 15:48:12.180385 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:48:12.180385 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:48:12.180385 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:48:12.180385 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:48:12.180385 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:48:12.180385 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:48:12.180385 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:48:12.180385 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Jun 23 15:48:12.180385 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:48:12.180385 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:48:12.180385 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:48:12.180385 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:48:12.181231 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:48:12.181231 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:48:12.181231 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:48:12.181231 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:48:12.181231 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 15:48:12.181231 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 15:48:12.181231 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 15:48:12.181231 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 15:48:12.181231 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 15:48:12.181231 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 15:48:12.181231 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 15:48:12.181231 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Jun 23 15:48:12.187662 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 349/722] 10.222.0.46 () {56 vars in 1096 bytes} [Thu Jun 23 15:48:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:48:12.195785 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476ea5d6-df90-46e3-a9f8-354b5fee64d6 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:12.347319 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476ea5d6-df90-46e3-a9f8-354b5fee64d6 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:12.394469 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ed50a6-1c99-4045-a120-3bbbfd71eaa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:12.395143 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ed50a6-1c99-4045-a120-3bbbfd71eaa9 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:12.395424 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ed50a6-1c99-4045-a120-3bbbfd71eaa9 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:12.395726 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ed50a6-1c99-4045-a120-3bbbfd71eaa9 None admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:12.396531 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ed50a6-1c99-4045-a120-3bbbfd71eaa9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:48:12.396841 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ed50a6-1c99-4045-a120-3bbbfd71eaa9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:12.398309 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ed50a6-1c99-4045-a120-3bbbfd71eaa9 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:12.407736 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 374/723] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Jun 23 15:48:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:48:12.413310 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b981bc-e80a-4f5e-87fa-0989fe15c8b0 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:12.512998 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:12.534703 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b981bc-e80a-4f5e-87fa-0989fe15c8b0 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:12.578460 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2d4b8e-4ac0-41a0-ba88-df2a73c6f942 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:12.579536 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2d4b8e-4ac0-41a0-ba88-df2a73c6f942 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:12.579992 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2d4b8e-4ac0-41a0-ba88-df2a73c6f942 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:12.580464 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2d4b8e-4ac0-41a0-ba88-df2a73c6f942 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:12.591995 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2d4b8e-4ac0-41a0-ba88-df2a73c6f942 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:12.593706 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2d4b8e-4ac0-41a0-ba88-df2a73c6f942 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:12.611603 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-fb2d4b8e-4ac0-41a0-ba88-df2a73c6f942 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 15:48:12.611603 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:48:12.611603 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:48:12.611603 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:48:12.611603 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:48:12.611603 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:48:12.611603 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:48:12.611603 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:48:12.611603 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:48:12.611603 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:48:12.611603 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:48:12.611603 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:48:12.611603 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 15:48:12.611603 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 15:48:12.611603 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 15:48:12.612541 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 15:48:12.612541 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:48:12.612541 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:48:12.612541 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:48:12.612541 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:48:12.612541 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:48:12.612541 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:48:12.612541 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:48:12.612541 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Jun 23 15:48:12.612541 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:48:12.612541 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:48:12.612541 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:48:12.612541 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:48:12.612541 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:48:12.612541 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:48:12.612541 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:48:12.612541 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:48:12.613489 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 15:48:12.613489 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 15:48:12.613489 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 15:48:12.613489 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Jun 23 15:48:12.616163 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 350/724] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Jun 23 15:48:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:48:12.621849 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ed50a6-1c99-4045-a120-3bbbfd71eaa9 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:12.626060 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:12.736800 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ed50a6-1c99-4045-a120-3bbbfd71eaa9 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:12.771237 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865b4ef9-0f30-4141-8893-d8e2fcd34dc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:12.772010 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865b4ef9-0f30-4141-8893-d8e2fcd34dc4 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:12.772293 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865b4ef9-0f30-4141-8893-d8e2fcd34dc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:12.772567 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865b4ef9-0f30-4141-8893-d8e2fcd34dc4 None admin] PATH_INFO: `/v3/domains` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:12.773287 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865b4ef9-0f30-4141-8893-d8e2fcd34dc4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:48:12.773591 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865b4ef9-0f30-4141-8893-d8e2fcd34dc4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:12.775252 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865b4ef9-0f30-4141-8893-d8e2fcd34dc4 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:12.788080 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 375/725] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Jun 23 15:48:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:48:12.793556 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2d4b8e-4ac0-41a0-ba88-df2a73c6f942 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:12.912059 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449f4d64-5ebe-4c81-b343-7b15417f6db6 None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:12.913019 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449f4d64-5ebe-4c81-b343-7b15417f6db6 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:12.913521 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449f4d64-5ebe-4c81-b343-7b15417f6db6 None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:12.915411 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 376/726] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jun 23 15:48:12 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 15:48:12.920617 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a162dc9-0e8b-4348-8d25-33b0be2e370e None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:12.921138 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a162dc9-0e8b-4348-8d25-33b0be2e370e None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:12.921636 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a162dc9-0e8b-4348-8d25-33b0be2e370e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:12.930235 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2d4b8e-4ac0-41a0-ba88-df2a73c6f942 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:12.939928 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-7a162dc9-0e8b-4348-8d25-33b0be2e370e None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:48:12.941392 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-7a162dc9-0e8b-4348-8d25-33b0be2e370e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:48:12.964339 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6c9467-9d87-4636-82bc-93a78f078546 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:12.965508 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6c9467-9d87-4636-82bc-93a78f078546 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:12.965936 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6c9467-9d87-4636-82bc-93a78f078546 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:12.966454 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6c9467-9d87-4636-82bc-93a78f078546 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:12.975404 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6c9467-9d87-4636-82bc-93a78f078546 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:12.977251 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6c9467-9d87-4636-82bc-93a78f078546 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:12.983988 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:12.999973 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-2e6c9467-9d87-4636-82bc-93a78f078546 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 23 15:48:12.999973 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:48:12.999973 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:48:12.999973 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:48:12.999973 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:48:12.999973 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:48:12.999973 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:48:12.999973 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:48:12.999973 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:48:12.999973 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:48:12.999973 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:48:12.999973 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:48:12.999973 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 15:48:12.999973 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 15:48:12.999973 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 15:48:12.999973 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 15:48:13.001015 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:48:13.001015 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:48:13.001015 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 15:48:13.001015 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 15:48:13.001015 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 15:48:13.001015 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 15:48:13.001015 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:48:13.001015 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:48:13.001015 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:48:13.001015 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:48:13.001015 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:48:13.001015 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Jun 23 15:48:13.001015 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:48:13.001015 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:48:13.001015 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:48:13.001015 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:48:13.001015 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:48:13.001015 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:48:13.002055 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:48:13.002055 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:48:13.002055 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 15:48:13.002055 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 15:48:13.002055 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 15:48:13.002055 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 15:48:13.002055 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 15:48:13.002055 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 15:48:13.002055 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 23 15:48:13.002055 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Jun 23 15:48:13.004502 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 351/727] 10.222.0.46 () {56 vars in 1096 bytes} [Thu Jun 23 15:48:12 2022] GET /identity/v3/users/swift => generated 83 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:48:13.012622 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6c9467-9d87-4636-82bc-93a78f078546 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:13.015730 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:13.025800 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 377/728] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jun 23 15:48:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 15:48:13.049268 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a162dc9-0e8b-4348-8d25-33b0be2e370e None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:13.145019 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6c9467-9d87-4636-82bc-93a78f078546 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:13.152873 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a162dc9-0e8b-4348-8d25-33b0be2e370e None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:13.181134 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a75be3-6615-4cf1-b2b2-d39004b87b02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:13.182112 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a75be3-6615-4cf1-b2b2-d39004b87b02 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:13.182598 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a75be3-6615-4cf1-b2b2-d39004b87b02 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:13.183083 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a75be3-6615-4cf1-b2b2-d39004b87b02 None admin] PATH_INFO: `/v3/users` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:13.184355 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a75be3-6615-4cf1-b2b2-d39004b87b02 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:48:13.184863 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a75be3-6615-4cf1-b2b2-d39004b87b02 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:13.186685 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a75be3-6615-4cf1-b2b2-d39004b87b02 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:13.189027 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591bb53f-0adc-4c81-b7bf-3f2cb3254254 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:13.189707 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591bb53f-0adc-4c81-b7bf-3f2cb3254254 None admin] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:13.189994 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591bb53f-0adc-4c81-b7bf-3f2cb3254254 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:13.190274 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591bb53f-0adc-4c81-b7bf-3f2cb3254254 None admin] PATH_INFO: `/v3/services` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:13.191032 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591bb53f-0adc-4c81-b7bf-3f2cb3254254 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:13.192411 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591bb53f-0adc-4c81-b7bf-3f2cb3254254 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:13.214821 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 352/730] 10.222.0.46 () {56 vars in 1135 bytes} [Thu Jun 23 15:48:13 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:48:13.215024 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 378/730] 10.222.0.46 () {60 vars in 1141 bytes} [Thu Jun 23 15:48:13 2022] POST /identity/v3/services => generated 251 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 15:48:13.216639 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c282996-ff46-4c9d-9069-a3e2e5047ca9 None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:13.216996 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c282996-ff46-4c9d-9069-a3e2e5047ca9 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:13.217300 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c282996-ff46-4c9d-9069-a3e2e5047ca9 None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:13.218056 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a75be3-6615-4cf1-b2b2-d39004b87b02 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:13.219008 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 379/731] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jun 23 15:48:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 15:48:13.226467 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cfa0c1-3221-4935-9387-c466f9b1d871 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:13.226857 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cfa0c1-3221-4935-9387-c466f9b1d871 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:13.227156 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cfa0c1-3221-4935-9387-c466f9b1d871 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:13.273452 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-43cfa0c1-3221-4935-9387-c466f9b1d871 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:48:13.275293 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-43cfa0c1-3221-4935-9387-c466f9b1d871 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:48:13.299953 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:13.349724 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:13.359889 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a75be3-6615-4cf1-b2b2-d39004b87b02 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:13.384720 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 380/732] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jun 23 15:48:13 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 15:48:13.386201 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454f7b7b-63be-42de-80e1-07766437819d None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:13.386495 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454f7b7b-63be-42de-80e1-07766437819d None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:13.386835 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454f7b7b-63be-42de-80e1-07766437819d None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:13.388641 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 381/733] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jun 23 15:48:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 15:48:13.393704 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d71720b-0ab8-48c8-8be7-5dcfb3ae4476 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:13.393994 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d71720b-0ab8-48c8-8be7-5dcfb3ae4476 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:13.394276 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d71720b-0ab8-48c8-8be7-5dcfb3ae4476 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:13.413476 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-4d71720b-0ab8-48c8-8be7-5dcfb3ae4476 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:48:13.414651 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf0186e-cc86-466b-8a97-48cdf23b7092 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:13.414826 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-4d71720b-0ab8-48c8-8be7-5dcfb3ae4476 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:48:13.415402 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf0186e-cc86-466b-8a97-48cdf23b7092 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:13.415693 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf0186e-cc86-466b-8a97-48cdf23b7092 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:13.415978 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf0186e-cc86-466b-8a97-48cdf23b7092 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:13.498186 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 382/734] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jun 23 15:48:13 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 23 15:48:13.499441 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d71720b-0ab8-48c8-8be7-5dcfb3ae4476 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:13.504472 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf0186e-cc86-466b-8a97-48cdf23b7092 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:13.506000 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf0186e-cc86-466b-8a97-48cdf23b7092 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:13.586728 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:13.631535 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d71720b-0ab8-48c8-8be7-5dcfb3ae4476 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:13.643780 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:13.652302 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 353/735] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Jun 23 15:48:13 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 436 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 23 15:48:13.653755 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf0186e-cc86-466b-8a97-48cdf23b7092 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:13.672582 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc40016-70fa-437f-9ddd-fb25a3781f6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:13.673259 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc40016-70fa-437f-9ddd-fb25a3781f6b None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:13.673551 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc40016-70fa-437f-9ddd-fb25a3781f6b None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:13.673829 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc40016-70fa-437f-9ddd-fb25a3781f6b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:13.686034 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc40016-70fa-437f-9ddd-fb25a3781f6b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 15:48:13.694462 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-acc40016-70fa-437f-9ddd-fb25a3781f6b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 15:48:13.694462 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:48:13.694462 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:48:13.694462 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:48:13.694462 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:48:13.694462 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:48:13.694462 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:48:13.694462 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:48:13.694462 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:48:13.694462 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:48:13.694462 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:48:13.694462 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:48:13.694462 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 15:48:13.694462 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 15:48:13.694462 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 15:48:13.694462 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 15:48:13.695463 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 15:48:13.695463 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 15:48:13.695463 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 15:48:13.695463 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 15:48:13.695463 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:48:13.695463 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:48:13.695463 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:48:13.695463 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:48:13.695463 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:48:13.695463 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:48:13.695463 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:48:13.695463 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Jun 23 15:48:13.695463 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:48:13.695463 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:48:13.695463 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:48:13.695463 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:48:13.696313 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:48:13.696313 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:48:13.696313 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:48:13.696313 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:48:13.696313 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 15:48:13.696313 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 15:48:13.696313 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 15:48:13.696313 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 15:48:13.696313 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 15:48:13.696313 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 15:48:13.696313 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 15:48:13.696313 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Jun 23 15:48:13.696313 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 383/736] 10.222.0.46 () {56 vars in 1096 bytes} [Thu Jun 23 15:48:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:48:13.697034 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc40016-70fa-437f-9ddd-fb25a3781f6b None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:13.783779 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf0186e-cc86-466b-8a97-48cdf23b7092 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:13.806257 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc40016-70fa-437f-9ddd-fb25a3781f6b None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:13.820572 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23631c7e-9cf0-4e1d-942f-099f0c307b8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:13.821391 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23631c7e-9cf0-4e1d-942f-099f0c307b8b None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:13.821683 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23631c7e-9cf0-4e1d-942f-099f0c307b8b None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:13.821964 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23631c7e-9cf0-4e1d-942f-099f0c307b8b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:13.832917 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23631c7e-9cf0-4e1d-942f-099f0c307b8b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 15:48:13.841148 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-23631c7e-9cf0-4e1d-942f-099f0c307b8b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 23 15:48:13.841148 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:48:13.841148 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:48:13.841148 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:48:13.841148 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:48:13.841148 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:48:13.841148 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:48:13.841148 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:48:13.841148 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:48:13.841148 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:48:13.841148 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:48:13.841148 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:48:13.841148 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 15:48:13.841148 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 15:48:13.841148 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 15:48:13.841148 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 15:48:13.842095 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 15:48:13.842095 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 15:48:13.842095 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 15:48:13.842095 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 15:48:13.842095 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:48:13.842095 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:48:13.842095 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:48:13.842095 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:48:13.842095 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:48:13.842095 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:48:13.842095 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:48:13.842095 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Jun 23 15:48:13.842095 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:48:13.842095 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:48:13.842095 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:48:13.842095 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:48:13.843088 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:48:13.843088 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:48:13.843088 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:48:13.843088 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:48:13.843088 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 15:48:13.843088 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 15:48:13.843088 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 15:48:13.843088 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 15:48:13.843088 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 15:48:13.843088 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 15:48:13.843088 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 23 15:48:13.843088 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Jun 23 15:48:13.843088 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 354/737] 10.222.0.46 () {56 vars in 1102 bytes} [Thu Jun 23 15:48:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:48:13.843782 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23631c7e-9cf0-4e1d-942f-099f0c307b8b None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:13.854060 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75524a10-c40e-44db-bddd-4404da4125aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:13.854910 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75524a10-c40e-44db-bddd-4404da4125aa None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:13.855197 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75524a10-c40e-44db-bddd-4404da4125aa None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:13.855472 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75524a10-c40e-44db-bddd-4404da4125aa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:13.863710 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:13.868919 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75524a10-c40e-44db-bddd-4404da4125aa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:13.870522 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75524a10-c40e-44db-bddd-4404da4125aa None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:13.889662 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-75524a10-c40e-44db-bddd-4404da4125aa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 15:48:13.889662 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:48:13.889662 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:48:13.889662 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:48:13.889662 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:48:13.889662 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:48:13.889662 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:48:13.889662 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:48:13.889662 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:48:13.889662 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:48:13.889662 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:48:13.889662 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:48:13.889662 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 15:48:13.889662 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 15:48:13.889662 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 15:48:13.890617 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 15:48:13.890617 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:48:13.890617 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:48:13.890617 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:48:13.890617 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:48:13.890617 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:48:13.890617 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:48:13.890617 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:48:13.890617 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Jun 23 15:48:13.890617 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:48:13.890617 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:48:13.890617 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:48:13.890617 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:48:13.890617 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:48:13.890617 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:48:13.890617 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:48:13.890617 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:48:13.891554 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 15:48:13.891554 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 15:48:13.891554 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 15:48:13.891554 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Jun 23 15:48:13.891554 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 384/738] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Jun 23 15:48:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:48:13.892590 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504fe1b8-2768-42dc-be04-9f4e02eda118 None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:13.892590 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504fe1b8-2768-42dc-be04-9f4e02eda118 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:13.892779 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504fe1b8-2768-42dc-be04-9f4e02eda118 None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:13.894059 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 385/739] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jun 23 15:48:13 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 15:48:13.894626 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504fe1b8-2768-42dc-be04-9f4e02eda118 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:13.924171 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:13.966661 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23631c7e-9cf0-4e1d-942f-099f0c307b8b None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:14.001989 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf81f40-1a86-41bb-acd8-5e66ff56be50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:14.002683 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf81f40-1a86-41bb-acd8-5e66ff56be50 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:14.002998 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf81f40-1a86-41bb-acd8-5e66ff56be50 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:14.003279 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf81f40-1a86-41bb-acd8-5e66ff56be50 None admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:14.004049 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf81f40-1a86-41bb-acd8-5e66ff56be50 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:48:14.004347 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf81f40-1a86-41bb-acd8-5e66ff56be50 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:14.005847 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf81f40-1a86-41bb-acd8-5e66ff56be50 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:14.010229 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504fe1b8-2768-42dc-be04-9f4e02eda118 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:14.014506 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 355/740] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Jun 23 15:48:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:48:14.015811 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e158d17-a3da-4880-b074-c3511dbf6335 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:14.016105 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e158d17-a3da-4880-b074-c3511dbf6335 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:14.016386 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e158d17-a3da-4880-b074-c3511dbf6335 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:14.035352 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-5e158d17-a3da-4880-b074-c3511dbf6335 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:48:14.037114 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-5e158d17-a3da-4880-b074-c3511dbf6335 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:48:14.049583 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a62912-2c1a-4043-ae9e-77fc69f64960 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:14.050546 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a62912-2c1a-4043-ae9e-77fc69f64960 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:14.050997 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a62912-2c1a-4043-ae9e-77fc69f64960 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:14.051413 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a62912-2c1a-4043-ae9e-77fc69f64960 None admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:14.052335 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a62912-2c1a-4043-ae9e-77fc69f64960 None admin] RBAC: Adding query filter params (name=service) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:48:14.052782 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a62912-2c1a-4043-ae9e-77fc69f64960 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:14.054296 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a62912-2c1a-4043-ae9e-77fc69f64960 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:14.067225 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 386/741] 10.222.0.46 () {56 vars in 1103 bytes} [Thu Jun 23 15:48:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:48:14.068290 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a62912-2c1a-4043-ae9e-77fc69f64960 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:14.109303 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 356/742] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jun 23 15:48:14 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 23 15:48:14.110586 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e158d17-a3da-4880-b074-c3511dbf6335 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:14.124971 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:14.157926 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:14.176296 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a62912-2c1a-4043-ae9e-77fc69f64960 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:14.212227 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9d555b-9d3b-4913-8d7c-c9b0affe1dc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:14.212957 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9d555b-9d3b-4913-8d7c-c9b0affe1dc1 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:14.213234 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9d555b-9d3b-4913-8d7c-c9b0affe1dc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:14.213529 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9d555b-9d3b-4913-8d7c-c9b0affe1dc1 None admin] PATH_INFO: `/v3/domains` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:14.214256 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9d555b-9d3b-4913-8d7c-c9b0affe1dc1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:48:14.214564 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9d555b-9d3b-4913-8d7c-c9b0affe1dc1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:14.216126 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9d555b-9d3b-4913-8d7c-c9b0affe1dc1 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:14.227149 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e158d17-a3da-4880-b074-c3511dbf6335 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:14.228631 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 387/743] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Jun 23 15:48:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:48:14.233708 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9d555b-9d3b-4913-8d7c-c9b0affe1dc1 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:14.265505 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8002b521-e179-48ba-aeec-717fce3b53a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:14.266340 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8002b521-e179-48ba-aeec-717fce3b53a6 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:14.266627 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8002b521-e179-48ba-aeec-717fce3b53a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:14.266930 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8002b521-e179-48ba-aeec-717fce3b53a6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:14.278317 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8002b521-e179-48ba-aeec-717fce3b53a6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:14.279780 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8002b521-e179-48ba-aeec-717fce3b53a6 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:14.299083 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-8002b521-e179-48ba-aeec-717fce3b53a6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 15:48:14.299083 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:48:14.299083 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:48:14.299083 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:48:14.299083 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:48:14.299083 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:48:14.299083 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:48:14.299083 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:48:14.299083 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:48:14.299083 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:48:14.299083 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:48:14.299083 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:48:14.299083 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 15:48:14.299083 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 15:48:14.299083 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 15:48:14.299981 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 15:48:14.299981 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:48:14.299981 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:48:14.299981 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:48:14.299981 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:48:14.299981 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:48:14.299981 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:48:14.299981 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:48:14.299981 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Jun 23 15:48:14.299981 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:48:14.299981 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:48:14.299981 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:48:14.299981 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:48:14.299981 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:48:14.299981 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:48:14.299981 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:48:14.299981 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:48:14.300860 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 15:48:14.300860 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 15:48:14.300860 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 15:48:14.300860 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Jun 23 15:48:14.300860 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 357/744] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Jun 23 15:48:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:48:14.301144 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8002b521-e179-48ba-aeec-717fce3b53a6 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:14.337592 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9d555b-9d3b-4913-8d7c-c9b0affe1dc1 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:14.363676 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:14.374444 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccceb4c5-2c43-4568-a6b3-c7386a7e4a76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:14.375358 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccceb4c5-2c43-4568-a6b3-c7386a7e4a76 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:14.375713 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccceb4c5-2c43-4568-a6b3-c7386a7e4a76 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:14.376061 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccceb4c5-2c43-4568-a6b3-c7386a7e4a76 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:14.389077 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccceb4c5-2c43-4568-a6b3-c7386a7e4a76 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:14.390628 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccceb4c5-2c43-4568-a6b3-c7386a7e4a76 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:14.393627 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:14.407699 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8002b521-e179-48ba-aeec-717fce3b53a6 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:14.410764 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-ccceb4c5-2c43-4568-a6b3-c7386a7e4a76 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 15:48:14.410764 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:48:14.410764 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:48:14.410764 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:48:14.410764 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:48:14.410764 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:48:14.410764 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:48:14.410764 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:48:14.410764 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:48:14.410764 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:48:14.410764 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:48:14.410764 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:48:14.410764 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 15:48:14.410764 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 15:48:14.410764 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 15:48:14.412344 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 15:48:14.412344 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:48:14.412344 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:48:14.412344 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:48:14.412344 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:48:14.412344 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:48:14.412344 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:48:14.412344 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:48:14.412344 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Jun 23 15:48:14.412344 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:48:14.412344 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:48:14.412344 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:48:14.412344 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:48:14.412344 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:48:14.412344 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:48:14.412344 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:48:14.412344 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:48:14.413463 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 15:48:14.413463 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 15:48:14.413463 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 15:48:14.413463 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Jun 23 15:48:14.414494 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 388/745] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Jun 23 15:48:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:48:14.419523 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccceb4c5-2c43-4568-a6b3-c7386a7e4a76 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:14.446938 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8597abaf-871b-40f2-8385-da9d8b3ddef0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:14.447761 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8597abaf-871b-40f2-8385-da9d8b3ddef0 None admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:14.448137 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8597abaf-871b-40f2-8385-da9d8b3ddef0 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:14.448508 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8597abaf-871b-40f2-8385-da9d8b3ddef0 None admin] PATH_INFO: `/v3/services` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:14.449328 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8597abaf-871b-40f2-8385-da9d8b3ddef0 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:14.450908 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8597abaf-871b-40f2-8385-da9d8b3ddef0 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:14.466197 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 358/746] 10.222.0.46 () {60 vars in 1141 bytes} [Thu Jun 23 15:48:14 2022] POST /identity/v3/services => generated 240 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 15:48:14.467070 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8597abaf-871b-40f2-8385-da9d8b3ddef0 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:14.557418 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccceb4c5-2c43-4568-a6b3-c7386a7e4a76 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:14.602518 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c7d052-77f8-4934-b653-73deea9d5dff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:14.603519 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c7d052-77f8-4934-b653-73deea9d5dff None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:14.603908 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c7d052-77f8-4934-b653-73deea9d5dff None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:14.604290 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c7d052-77f8-4934-b653-73deea9d5dff None admin] PATH_INFO: `/v3/projects/service` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:14.611798 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8597abaf-871b-40f2-8385-da9d8b3ddef0 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:14.612400 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c7d052-77f8-4934-b653-73deea9d5dff None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:14.616132 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c7d052-77f8-4934-b653-73deea9d5dff None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:14.628952 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:14.638926 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-24c7d052-77f8-4934-b653-73deea9d5dff None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 15:48:14.638926 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:48:14.638926 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:48:14.638926 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:48:14.638926 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:48:14.638926 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:48:14.638926 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:48:14.638926 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:48:14.638926 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:48:14.638926 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:48:14.638926 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:48:14.638926 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:48:14.638926 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 15:48:14.638926 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 15:48:14.638926 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 15:48:14.640087 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 15:48:14.640087 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:48:14.640087 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:48:14.640087 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:48:14.640087 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:48:14.640087 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:48:14.640087 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:48:14.640087 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:48:14.640087 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Jun 23 15:48:14.640087 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:48:14.640087 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:48:14.640087 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:48:14.640087 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:48:14.640087 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:48:14.640087 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:48:14.640087 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:48:14.640087 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:48:14.641419 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 15:48:14.641419 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 15:48:14.641419 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 15:48:14.641419 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 15:48:14.641419 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 15:48:14.641419 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 15:48:14.641419 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 15:48:14.641419 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Jun 23 15:48:14.641419 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 389/747] 10.222.0.46 () {56 vars in 1111 bytes} [Thu Jun 23 15:48:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:48:14.641419 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c7d052-77f8-4934-b653-73deea9d5dff None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:14.653618 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7961c84-e626-474c-95af-1eeab3c3a6cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:14.654408 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7961c84-e626-474c-95af-1eeab3c3a6cd None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:14.654721 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7961c84-e626-474c-95af-1eeab3c3a6cd None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:14.655061 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7961c84-e626-474c-95af-1eeab3c3a6cd None admin] PATH_INFO: `/v3/domains` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:14.655913 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7961c84-e626-474c-95af-1eeab3c3a6cd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:48:14.656227 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7961c84-e626-474c-95af-1eeab3c3a6cd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:14.657674 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7961c84-e626-474c-95af-1eeab3c3a6cd None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:14.671260 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 359/748] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Jun 23 15:48:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:48:14.672121 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7961c84-e626-474c-95af-1eeab3c3a6cd None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:14.700618 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:14.769567 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c7d052-77f8-4934-b653-73deea9d5dff None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:14.789258 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7961c84-e626-474c-95af-1eeab3c3a6cd None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:14.817448 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac71883-81fd-4fd5-b509-04468da0cc83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:14.818952 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac71883-81fd-4fd5-b509-04468da0cc83 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:14.819237 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac71883-81fd-4fd5-b509-04468da0cc83 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:14.819956 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac71883-81fd-4fd5-b509-04468da0cc83 None admin] PATH_INFO: `/v3/domains` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:14.820982 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac71883-81fd-4fd5-b509-04468da0cc83 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:48:14.821302 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac71883-81fd-4fd5-b509-04468da0cc83 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:14.823574 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac71883-81fd-4fd5-b509-04468da0cc83 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:14.828231 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94819553-6b33-404e-8a86-4adfe6d1c8d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:14.829050 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94819553-6b33-404e-8a86-4adfe6d1c8d7 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:14.829405 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94819553-6b33-404e-8a86-4adfe6d1c8d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:14.829759 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94819553-6b33-404e-8a86-4adfe6d1c8d7 None admin] PATH_INFO: `/v3/projects` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:14.830634 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94819553-6b33-404e-8a86-4adfe6d1c8d7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:48:14.831130 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94819553-6b33-404e-8a86-4adfe6d1c8d7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:14.832656 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94819553-6b33-404e-8a86-4adfe6d1c8d7 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:14.842787 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 390/749] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Jun 23 15:48:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:48:14.843693 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac71883-81fd-4fd5-b509-04468da0cc83 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:14.849586 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 360/750] 10.222.0.46 () {56 vars in 1148 bytes} [Thu Jun 23 15:48:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:48:14.850517 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94819553-6b33-404e-8a86-4adfe6d1c8d7 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:14.889144 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:14.963041 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac71883-81fd-4fd5-b509-04468da0cc83 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:14.977269 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94819553-6b33-404e-8a86-4adfe6d1c8d7 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:14.978115 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:15.002909 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbebf9ac-c789-4482-b39a-6702fc63a123 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:15.004119 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbebf9ac-c789-4482-b39a-6702fc63a123 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:15.004609 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbebf9ac-c789-4482-b39a-6702fc63a123 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:15.007024 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbebf9ac-c789-4482-b39a-6702fc63a123 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:15.020026 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbebf9ac-c789-4482-b39a-6702fc63a123 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:15.022637 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbebf9ac-c789-4482-b39a-6702fc63a123 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:15.027540 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b923fbcb-40b1-449c-a48a-85780fa412b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:15.028643 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b923fbcb-40b1-449c-a48a-85780fa412b6 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:15.028939 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b923fbcb-40b1-449c-a48a-85780fa412b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:15.029224 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b923fbcb-40b1-449c-a48a-85780fa412b6 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:15.037915 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b923fbcb-40b1-449c-a48a-85780fa412b6 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:15.039427 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b923fbcb-40b1-449c-a48a-85780fa412b6 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:15.040639 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-bbebf9ac-c789-4482-b39a-6702fc63a123 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 23 15:48:15.040639 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:48:15.040639 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:48:15.040639 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:48:15.040639 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:48:15.040639 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:48:15.040639 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:48:15.040639 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:48:15.040639 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:48:15.040639 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:48:15.040639 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:48:15.040639 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:48:15.040639 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 15:48:15.040639 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 15:48:15.040639 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 15:48:15.040639 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 15:48:15.041552 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:48:15.041552 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:48:15.041552 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 15:48:15.041552 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 15:48:15.041552 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 15:48:15.041552 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 15:48:15.041552 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:48:15.041552 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:48:15.041552 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:48:15.041552 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:48:15.041552 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:48:15.041552 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Jun 23 15:48:15.041552 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:48:15.041552 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:48:15.041552 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:48:15.041552 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:48:15.041552 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:48:15.041552 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:48:15.042481 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:48:15.042481 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:48:15.042481 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 15:48:15.042481 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 15:48:15.042481 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 15:48:15.042481 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 15:48:15.042481 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 15:48:15.042481 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 15:48:15.042481 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 23 15:48:15.042481 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Jun 23 15:48:15.045169 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 391/751] 10.222.0.46 () {56 vars in 1093 bytes} [Thu Jun 23 15:48:14 2022] GET /identity/v3/users/nova => generated 82 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:48:15.046576 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbebf9ac-c789-4482-b39a-6702fc63a123 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:15.055775 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-b923fbcb-40b1-449c-a48a-85780fa412b6 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 23 15:48:15.055775 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:48:15.055775 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:48:15.055775 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:48:15.055775 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:48:15.055775 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:48:15.055775 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:48:15.055775 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:48:15.055775 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:48:15.055775 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:48:15.055775 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:48:15.055775 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:48:15.055775 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 15:48:15.055775 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 15:48:15.055775 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 15:48:15.055775 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 15:48:15.056777 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:48:15.056777 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:48:15.056777 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 15:48:15.056777 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 15:48:15.056777 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 15:48:15.056777 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 15:48:15.056777 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:48:15.056777 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:48:15.056777 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:48:15.056777 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:48:15.056777 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:48:15.056777 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Jun 23 15:48:15.056777 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:48:15.056777 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:48:15.056777 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:48:15.056777 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:48:15.056777 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:48:15.056777 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:48:15.057763 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:48:15.057763 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:48:15.057763 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 15:48:15.057763 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 15:48:15.057763 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 15:48:15.057763 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 15:48:15.057763 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 15:48:15.057763 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 15:48:15.057763 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 23 15:48:15.057763 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Jun 23 15:48:15.060640 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 361/752] 10.222.0.46 () {56 vars in 1117 bytes} [Thu Jun 23 15:48:14 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:48:15.061817 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b923fbcb-40b1-449c-a48a-85780fa412b6 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:15.167853 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:15.175672 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b923fbcb-40b1-449c-a48a-85780fa412b6 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:15.184797 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbebf9ac-c789-4482-b39a-6702fc63a123 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:15.213578 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d612cc-8647-4894-815f-05f31af6d043 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:15.214639 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d612cc-8647-4894-815f-05f31af6d043 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:15.215335 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d612cc-8647-4894-815f-05f31af6d043 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:15.215842 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d612cc-8647-4894-815f-05f31af6d043 None admin] PATH_INFO: `/v3/users` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:15.216945 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d612cc-8647-4894-815f-05f31af6d043 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:48:15.217463 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d612cc-8647-4894-815f-05f31af6d043 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:15.219157 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d612cc-8647-4894-815f-05f31af6d043 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:15.230276 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abee87c3-406a-4d2d-b5a4-5aa60a6e791a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:15.231356 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abee87c3-406a-4d2d-b5a4-5aa60a6e791a None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:15.231852 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abee87c3-406a-4d2d-b5a4-5aa60a6e791a None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:15.232344 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abee87c3-406a-4d2d-b5a4-5aa60a6e791a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:15.233390 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abee87c3-406a-4d2d-b5a4-5aa60a6e791a None admin] RBAC: Adding query filter params (role.id=812d3c567c52469cb2b35709fbdc857c, scope.project.id=28ecdf5b474c44098c722f677d2a0b77, user.id=242bf02d14724fe4837fcce8730377f9) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:48:15.233909 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abee87c3-406a-4d2d-b5a4-5aa60a6e791a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:15.235742 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abee87c3-406a-4d2d-b5a4-5aa60a6e791a None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:15.241493 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 362/753] 10.222.0.46 () {56 vars in 1133 bytes} [Thu Jun 23 15:48:15 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:48:15.242383 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d612cc-8647-4894-815f-05f31af6d043 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:15.243722 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 392/754] 10.222.0.46 () {56 vars in 1374 bytes} [Thu Jun 23 15:48:15 2022] GET /identity/v3/role_assignments?user.id=242bf02d14724fe4837fcce8730377f9&scope.project.id=28ecdf5b474c44098c722f677d2a0b77&role.id=812d3c567c52469cb2b35709fbdc857c => generated 259 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:48:15.246460 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abee87c3-406a-4d2d-b5a4-5aa60a6e791a None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:15.315280 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:15.399176 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abee87c3-406a-4d2d-b5a4-5aa60a6e791a None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:15.448299 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0151635d-53dd-403c-9f16-1f4c2be50e7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:15.449794 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0151635d-53dd-403c-9f16-1f4c2be50e7e None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:15.450263 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0151635d-53dd-403c-9f16-1f4c2be50e7e None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:15.450770 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0151635d-53dd-403c-9f16-1f4c2be50e7e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:15.467786 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d612cc-8647-4894-815f-05f31af6d043 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:15.479525 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0151635d-53dd-403c-9f16-1f4c2be50e7e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:15.481285 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0151635d-53dd-403c-9f16-1f4c2be50e7e None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:15.518333 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-0151635d-53dd-403c-9f16-1f4c2be50e7e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 15:48:15.518333 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:48:15.518333 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:48:15.518333 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:48:15.518333 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:48:15.518333 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:48:15.518333 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:48:15.518333 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:48:15.518333 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:48:15.518333 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:48:15.518333 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:48:15.518333 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:48:15.518333 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 15:48:15.518333 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 15:48:15.518333 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 15:48:15.519310 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 15:48:15.519310 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:48:15.519310 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:48:15.519310 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:48:15.519310 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:48:15.519310 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:48:15.519310 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:48:15.519310 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:48:15.519310 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Jun 23 15:48:15.519310 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:48:15.519310 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:48:15.519310 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:48:15.519310 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:48:15.519310 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:48:15.519310 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:48:15.519310 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:48:15.519310 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:48:15.520226 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 15:48:15.520226 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 15:48:15.520226 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 15:48:15.520226 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Jun 23 15:48:15.523296 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 393/755] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Jun 23 15:48:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 15:48:15.525032 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5930eb-7394-4944-b8b0-084ba6704446 None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:15.525562 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5930eb-7394-4944-b8b0-084ba6704446 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:15.526074 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5930eb-7394-4944-b8b0-084ba6704446 None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:15.527912 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 394/756] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jun 23 15:48:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 15:48:15.528808 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5930eb-7394-4944-b8b0-084ba6704446 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:15.539935 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead144de-5c03-4431-88b3-818d2f32482d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:15.540669 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead144de-5c03-4431-88b3-818d2f32482d None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:15.541061 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead144de-5c03-4431-88b3-818d2f32482d None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:15.541443 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead144de-5c03-4431-88b3-818d2f32482d None admin] PATH_INFO: `/v3/users` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:15.542669 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead144de-5c03-4431-88b3-818d2f32482d None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:48:15.542974 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead144de-5c03-4431-88b3-818d2f32482d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:15.544634 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead144de-5c03-4431-88b3-818d2f32482d None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:15.565839 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 363/757] 10.222.0.46 () {56 vars in 1149 bytes} [Thu Jun 23 15:48:15 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:48:15.570738 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d450dc5e-82bd-4424-aa2d-f17da8e8ea45 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:15.570738 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d450dc5e-82bd-4424-aa2d-f17da8e8ea45 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:15.570738 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d450dc5e-82bd-4424-aa2d-f17da8e8ea45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:15.589201 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-d450dc5e-82bd-4424-aa2d-f17da8e8ea45 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:48:15.590554 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-d450dc5e-82bd-4424-aa2d-f17da8e8ea45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:48:15.593945 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:15.638662 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:15.653354 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5930eb-7394-4944-b8b0-084ba6704446 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:15.678976 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 364/758] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jun 23 15:48:15 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 15:48:15.680369 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d450dc5e-82bd-4424-aa2d-f17da8e8ea45 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:15.694756 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26021d29-85a1-48c9-bc63-736901583ad8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:15.695448 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26021d29-85a1-48c9-bc63-736901583ad8 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:15.696019 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26021d29-85a1-48c9-bc63-736901583ad8 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:15.696407 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26021d29-85a1-48c9-bc63-736901583ad8 None admin] PATH_INFO: `/v3/domains` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:15.697277 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26021d29-85a1-48c9-bc63-736901583ad8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:48:15.697710 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26021d29-85a1-48c9-bc63-736901583ad8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:15.699731 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26021d29-85a1-48c9-bc63-736901583ad8 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:15.714257 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 395/759] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Jun 23 15:48:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:48:15.717762 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26021d29-85a1-48c9-bc63-736901583ad8 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:15.828164 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d450dc5e-82bd-4424-aa2d-f17da8e8ea45 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:15.836213 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26021d29-85a1-48c9-bc63-736901583ad8 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:15.872464 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5dcbac3-1e97-4fb0-8472-8e0e824897c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:15.873618 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dcbac3-1e97-4fb0-8472-8e0e824897c3 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:15.874111 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dcbac3-1e97-4fb0-8472-8e0e824897c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:15.874635 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dcbac3-1e97-4fb0-8472-8e0e824897c3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:15.875169 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4395d6de-b2fe-4b46-95d5-78e026e9000d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:15.875982 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4395d6de-b2fe-4b46-95d5-78e026e9000d None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:15.876264 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4395d6de-b2fe-4b46-95d5-78e026e9000d None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:15.876539 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4395d6de-b2fe-4b46-95d5-78e026e9000d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:15.883933 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4395d6de-b2fe-4b46-95d5-78e026e9000d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:15.885399 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4395d6de-b2fe-4b46-95d5-78e026e9000d None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:15.890641 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:15.903434 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-4395d6de-b2fe-4b46-95d5-78e026e9000d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 15:48:15.903434 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:48:15.903434 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:48:15.903434 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:48:15.903434 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:48:15.903434 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:48:15.903434 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:48:15.903434 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:48:15.903434 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:48:15.903434 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:48:15.903434 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:48:15.903434 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:48:15.903434 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 15:48:15.903434 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 15:48:15.903434 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 15:48:15.904763 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 15:48:15.904763 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:48:15.904763 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:48:15.904763 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:48:15.904763 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:48:15.904763 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:48:15.904763 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:48:15.904763 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:48:15.904763 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Jun 23 15:48:15.904763 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:48:15.904763 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:48:15.904763 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:48:15.904763 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:48:15.904763 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:48:15.904763 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:48:15.904763 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:48:15.904763 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:48:15.907243 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 15:48:15.907243 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 15:48:15.907243 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 15:48:15.907243 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 15:48:15.907243 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 15:48:15.907243 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 15:48:15.907243 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 15:48:15.907243 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Jun 23 15:48:15.907243 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 396/760] 10.222.0.46 () {56 vars in 1111 bytes} [Thu Jun 23 15:48:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:48:15.907243 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4395d6de-b2fe-4b46-95d5-78e026e9000d None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:15.917802 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:15.978676 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5dcbac3-1e97-4fb0-8472-8e0e824897c3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:15.983008 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5dcbac3-1e97-4fb0-8472-8e0e824897c3 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:16.024039 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4395d6de-b2fe-4b46-95d5-78e026e9000d None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:16.061407 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9a3aa9-0b25-495e-abff-9e0d9cf4c4e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:16.062559 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9a3aa9-0b25-495e-abff-9e0d9cf4c4e6 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:16.062774 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9a3aa9-0b25-495e-abff-9e0d9cf4c4e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:16.063148 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9a3aa9-0b25-495e-abff-9e0d9cf4c4e6 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:16.065246 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9a3aa9-0b25-495e-abff-9e0d9cf4c4e6 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:16.065966 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9a3aa9-0b25-495e-abff-9e0d9cf4c4e6 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:16.081468 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-5f9a3aa9-0b25-495e-abff-9e0d9cf4c4e6 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 23 15:48:16.081468 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:48:16.081468 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 15:48:16.081468 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 15:48:16.081468 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 15:48:16.081468 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 15:48:16.081468 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 15:48:16.081468 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 15:48:16.081468 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 23 15:48:16.081468 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Jun 23 15:48:16.081468 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 15:48:16.081468 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Jun 23 15:48:16.081468 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:48:16.081468 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:48:16.081468 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:48:16.081468 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:48:16.082715 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:48:16.082715 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:48:16.082715 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:48:16.082715 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:48:16.082715 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:48:16.082715 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:48:16.082715 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:48:16.082715 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 15:48:16.082715 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 15:48:16.082715 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 15:48:16.082715 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 15:48:16.082715 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:48:16.082715 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:48:16.082715 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:48:16.082715 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:48:16.082715 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:48:16.084155 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:48:16.084155 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:48:16.084155 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Jun 23 15:48:16.084155 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:48:16.084155 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:48:16.084155 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:48:16.084155 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:48:16.084155 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:48:16.084155 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:48:16.084155 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:48:16.084155 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:48:16.084155 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 15:48:16.084155 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 15:48:16.084155 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 23 15:48:16.084155 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Jun 23 15:48:16.089808 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 397/761] 10.222.0.46 () {56 vars in 1129 bytes} [Thu Jun 23 15:48:15 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:48:16.090163 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9a3aa9-0b25-495e-abff-9e0d9cf4c4e6 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:16.140839 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 365/762] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Jun 23 15:48:15 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 457 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 23 15:48:16.142041 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5dcbac3-1e97-4fb0-8472-8e0e824897c3 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:16.200459 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9a3aa9-0b25-495e-abff-9e0d9cf4c4e6 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:16.203319 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:16.204072 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:16.242553 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3953b3f0-ac8a-4451-939a-2f626156561c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:16.243272 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3953b3f0-ac8a-4451-939a-2f626156561c None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:16.243619 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3953b3f0-ac8a-4451-939a-2f626156561c None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:16.243965 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3953b3f0-ac8a-4451-939a-2f626156561c None admin] PATH_INFO: `/v3/projects` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:16.244890 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3953b3f0-ac8a-4451-939a-2f626156561c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:48:16.245268 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3953b3f0-ac8a-4451-939a-2f626156561c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:16.246830 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3953b3f0-ac8a-4451-939a-2f626156561c None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:16.261469 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 398/763] 10.222.0.46 () {56 vars in 1148 bytes} [Thu Jun 23 15:48:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:48:16.263326 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3953b3f0-ac8a-4451-939a-2f626156561c None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:16.283195 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5dcbac3-1e97-4fb0-8472-8e0e824897c3 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:16.334596 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151fce4e-d5cd-4625-ad3a-c30d3f927ceb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:16.336278 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151fce4e-d5cd-4625-ad3a-c30d3f927ceb None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:16.337054 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151fce4e-d5cd-4625-ad3a-c30d3f927ceb None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:16.337804 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151fce4e-d5cd-4625-ad3a-c30d3f927ceb None admin] PATH_INFO: `/v3/services` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:16.339032 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151fce4e-d5cd-4625-ad3a-c30d3f927ceb None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:48:16.339557 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151fce4e-d5cd-4625-ad3a-c30d3f927ceb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:16.341169 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151fce4e-d5cd-4625-ad3a-c30d3f927ceb None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:16.348064 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 366/764] 10.222.0.46 () {56 vars in 1124 bytes} [Thu Jun 23 15:48:16 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:48:16.352374 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151fce4e-d5cd-4625-ad3a-c30d3f927ceb None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:16.391359 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3953b3f0-ac8a-4451-939a-2f626156561c None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:16.430750 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1edd22-dd05-4388-ba1d-e69a1f726d6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:16.431900 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1edd22-dd05-4388-ba1d-e69a1f726d6c None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:16.432285 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1edd22-dd05-4388-ba1d-e69a1f726d6c None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:16.432653 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1edd22-dd05-4388-ba1d-e69a1f726d6c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:16.435267 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:16.446723 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1edd22-dd05-4388-ba1d-e69a1f726d6c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:16.448686 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1edd22-dd05-4388-ba1d-e69a1f726d6c None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:16.471167 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-5e1edd22-dd05-4388-ba1d-e69a1f726d6c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 15:48:16.471167 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:48:16.471167 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:48:16.471167 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:48:16.471167 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:48:16.471167 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:48:16.471167 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:48:16.471167 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:48:16.471167 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:48:16.471167 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:48:16.471167 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:48:16.471167 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:48:16.471167 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 15:48:16.471167 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 15:48:16.471167 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 15:48:16.472154 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 15:48:16.472154 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:48:16.472154 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:48:16.472154 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:48:16.472154 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:48:16.472154 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:48:16.472154 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:48:16.472154 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:48:16.472154 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Jun 23 15:48:16.472154 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:48:16.472154 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:48:16.472154 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:48:16.472154 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:48:16.472154 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:48:16.472154 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:48:16.472154 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:48:16.472154 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:48:16.473044 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 15:48:16.473044 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 15:48:16.473044 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 15:48:16.473044 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Jun 23 15:48:16.477309 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 399/765] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Jun 23 15:48:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:48:16.478386 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1edd22-dd05-4388-ba1d-e69a1f726d6c None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:16.486169 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151fce4e-d5cd-4625-ad3a-c30d3f927ceb None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:16.489186 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:16.525005 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0660d864-5865-4bd9-9439-cc54ac464eb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:16.525851 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0660d864-5865-4bd9-9439-cc54ac464eb5 None admin] REQUEST_METHOD: `PUT` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:16.526141 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0660d864-5865-4bd9-9439-cc54ac464eb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:16.526423 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0660d864-5865-4bd9-9439-cc54ac464eb5 None admin] PATH_INFO: `/v3/projects/28ecdf5b474c44098c722f677d2a0b77/users/705115d19bab424d894bea8b1b48a7f0/roles/812d3c567c52469cb2b35709fbdc857c` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:16.528498 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0660d864-5865-4bd9-9439-cc54ac464eb5 None admin] RBAC: Authorizing `identity:create_grant(project_id=28ecdf5b474c44098c722f677d2a0b77, user_id=705115d19bab424d894bea8b1b48a7f0, role_id=812d3c567c52469cb2b35709fbdc857c)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:16.529968 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0660d864-5865-4bd9-9439-cc54ac464eb5 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:16.561305 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 367/766] 10.222.0.46 () {58 vars in 1439 bytes} [Thu Jun 23 15:48:16 2022] PUT /identity/v3/projects/28ecdf5b474c44098c722f677d2a0b77/users/705115d19bab424d894bea8b1b48a7f0/roles/812d3c567c52469cb2b35709fbdc857c => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 15:48:16.562440 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0660d864-5865-4bd9-9439-cc54ac464eb5 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:16.679024 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1edd22-dd05-4388-ba1d-e69a1f726d6c None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:16.739797 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1e1a2c-90e7-4972-bfd8-eefd053349b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:16.740446 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1e1a2c-90e7-4972-bfd8-eefd053349b5 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:16.740732 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1e1a2c-90e7-4972-bfd8-eefd053349b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:16.741005 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1e1a2c-90e7-4972-bfd8-eefd053349b5 None admin] PATH_INFO: `/v3/services` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:16.741734 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1e1a2c-90e7-4972-bfd8-eefd053349b5 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:48:16.742044 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1e1a2c-90e7-4972-bfd8-eefd053349b5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:16.743691 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1e1a2c-90e7-4972-bfd8-eefd053349b5 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:16.749656 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 400/767] 10.222.0.46 () {56 vars in 1124 bytes} [Thu Jun 23 15:48:16 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:48:16.754950 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e868be2-d243-4e3a-82a8-a134e26b744c None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:16.754950 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e868be2-d243-4e3a-82a8-a134e26b744c None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:16.754950 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e868be2-d243-4e3a-82a8-a134e26b744c None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:16.757885 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 401/768] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jun 23 15:48:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 15:48:16.758922 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e868be2-d243-4e3a-82a8-a134e26b744c None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:16.796475 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0660d864-5865-4bd9-9439-cc54ac464eb5 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:16.826925 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:16.849230 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c013e0c-cbbd-4b46-a4c4-efe2ca56108a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:16.850176 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c013e0c-cbbd-4b46-a4c4-efe2ca56108a None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:16.850469 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c013e0c-cbbd-4b46-a4c4-efe2ca56108a None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:16.850800 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c013e0c-cbbd-4b46-a4c4-efe2ca56108a None admin] PATH_INFO: `/v3/domains` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:16.852158 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c013e0c-cbbd-4b46-a4c4-efe2ca56108a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:48:16.852489 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c013e0c-cbbd-4b46-a4c4-efe2ca56108a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:16.854166 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c013e0c-cbbd-4b46-a4c4-efe2ca56108a None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:16.874833 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 368/769] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Jun 23 15:48:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:48:16.876540 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ed46c4-0237-4751-90e4-aa1f5bf32deb None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:16.876832 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ed46c4-0237-4751-90e4-aa1f5bf32deb None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:16.877138 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ed46c4-0237-4751-90e4-aa1f5bf32deb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:16.898839 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-c7ed46c4-0237-4751-90e4-aa1f5bf32deb None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:48:16.900316 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-c7ed46c4-0237-4751-90e4-aa1f5bf32deb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:48:16.900636 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e868be2-d243-4e3a-82a8-a134e26b744c None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:16.927567 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:16.947624 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d45fcfa-3ba1-4690-a4b9-d954b65748a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:16.949149 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d45fcfa-3ba1-4690-a4b9-d954b65748a1 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:16.949537 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d45fcfa-3ba1-4690-a4b9-d954b65748a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:16.949917 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d45fcfa-3ba1-4690-a4b9-d954b65748a1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:16.951525 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d45fcfa-3ba1-4690-a4b9-d954b65748a1 None admin] RBAC: Adding query filter params (interface=public, service_id=a420e2d7b1d2411a9fd85a98587f8f7b, region_id=RegionOne) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:48:16.951966 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d45fcfa-3ba1-4690-a4b9-d954b65748a1 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:16.954227 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d45fcfa-3ba1-4690-a4b9-d954b65748a1 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:16.964883 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 402/770] 10.222.0.46 () {56 vars in 1251 bytes} [Thu Jun 23 15:48:16 2022] GET /identity/v3/endpoints?service_id=a420e2d7b1d2411a9fd85a98587f8f7b&interface=public®ion_id=RegionOne => generated 194 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:48:16.966437 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d45fcfa-3ba1-4690-a4b9-d954b65748a1 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:16.982829 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 369/771] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jun 23 15:48:16 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 15:48:16.983737 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ed46c4-0237-4751-90e4-aa1f5bf32deb None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:17.123838 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d45fcfa-3ba1-4690-a4b9-d954b65748a1 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:17.133882 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ed46c4-0237-4751-90e4-aa1f5bf32deb None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:17.138178 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:17.162485 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0553f0-21d8-43d0-bded-f1ff8a4d8022 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:17.163581 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0553f0-21d8-43d0-bded-f1ff8a4d8022 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:17.164012 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0553f0-21d8-43d0-bded-f1ff8a4d8022 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:17.164448 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0553f0-21d8-43d0-bded-f1ff8a4d8022 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:17.170880 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8621d3-ff34-4179-aa6b-1417590c0992 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:17.171565 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8621d3-ff34-4179-aa6b-1417590c0992 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:17.171853 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8621d3-ff34-4179-aa6b-1417590c0992 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:17.172134 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8621d3-ff34-4179-aa6b-1417590c0992 None admin] PATH_INFO: `/v3/services` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:17.172392 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0553f0-21d8-43d0-bded-f1ff8a4d8022 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:17.172827 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8621d3-ff34-4179-aa6b-1417590c0992 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:17.174193 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8621d3-ff34-4179-aa6b-1417590c0992 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:17.174311 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0553f0-21d8-43d0-bded-f1ff8a4d8022 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:17.180843 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 370/772] 10.222.0.46 () {56 vars in 1087 bytes} [Thu Jun 23 15:48:16 2022] GET /identity/v3/services => generated 772 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:48:17.181697 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8621d3-ff34-4179-aa6b-1417590c0992 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:17.190914 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-cb0553f0-21d8-43d0-bded-f1ff8a4d8022 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 15:48:17.190914 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:48:17.190914 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:48:17.190914 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:48:17.190914 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:48:17.190914 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:48:17.190914 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:48:17.190914 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:48:17.190914 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:48:17.190914 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:48:17.190914 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:48:17.190914 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:48:17.190914 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 15:48:17.190914 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 15:48:17.190914 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 15:48:17.191877 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 15:48:17.191877 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:48:17.191877 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:48:17.191877 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:48:17.191877 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:48:17.191877 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:48:17.191877 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:48:17.191877 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:48:17.191877 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Jun 23 15:48:17.191877 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:48:17.191877 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:48:17.191877 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:48:17.191877 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:48:17.191877 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:48:17.191877 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:48:17.191877 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:48:17.191877 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:48:17.192796 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 15:48:17.192796 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 15:48:17.192796 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 15:48:17.192796 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 15:48:17.192796 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 15:48:17.192796 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 15:48:17.192796 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 15:48:17.192796 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Jun 23 15:48:17.195532 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 403/773] 10.222.0.46 () {56 vars in 1111 bytes} [Thu Jun 23 15:48:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:48:17.203364 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:17.204730 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0553f0-21d8-43d0-bded-f1ff8a4d8022 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:17.358235 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0553f0-21d8-43d0-bded-f1ff8a4d8022 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:17.395130 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8621d3-ff34-4179-aa6b-1417590c0992 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:17.403211 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034c9ef7-a2de-4426-8250-b83ebfd5a7bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:17.403892 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034c9ef7-a2de-4426-8250-b83ebfd5a7bb None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:17.404178 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034c9ef7-a2de-4426-8250-b83ebfd5a7bb None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:17.404456 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034c9ef7-a2de-4426-8250-b83ebfd5a7bb None admin] PATH_INFO: `/v3/projects` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:17.405259 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034c9ef7-a2de-4426-8250-b83ebfd5a7bb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:48:17.406157 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034c9ef7-a2de-4426-8250-b83ebfd5a7bb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:17.407655 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034c9ef7-a2de-4426-8250-b83ebfd5a7bb None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:17.425481 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 404/774] 10.222.0.46 () {56 vars in 1148 bytes} [Thu Jun 23 15:48:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:48:17.433206 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034c9ef7-a2de-4426-8250-b83ebfd5a7bb None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:17.441664 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327999a8-7ca4-43a7-bdd6-8b1ad7df552c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:17.442558 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327999a8-7ca4-43a7-bdd6-8b1ad7df552c None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:17.442876 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327999a8-7ca4-43a7-bdd6-8b1ad7df552c None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:17.443202 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327999a8-7ca4-43a7-bdd6-8b1ad7df552c None admin] PATH_INFO: `/v3/services/network` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:17.443872 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327999a8-7ca4-43a7-bdd6-8b1ad7df552c None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:17.445343 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327999a8-7ca4-43a7-bdd6-8b1ad7df552c None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:17.457731 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-327999a8-7ca4-43a7-bdd6-8b1ad7df552c None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 23 15:48:17.457731 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:48:17.457731 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 15:48:17.457731 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 15:48:17.457731 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 15:48:17.457731 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 15:48:17.457731 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 15:48:17.457731 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 15:48:17.457731 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 23 15:48:17.457731 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Jun 23 15:48:17.457731 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 15:48:17.457731 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Jun 23 15:48:17.457731 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:48:17.457731 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:48:17.457731 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:48:17.457731 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:48:17.457731 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:48:17.458887 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:48:17.458887 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:48:17.458887 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:48:17.458887 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:48:17.458887 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:48:17.458887 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:48:17.458887 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 15:48:17.458887 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 15:48:17.458887 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 15:48:17.458887 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 15:48:17.458887 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:48:17.458887 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:48:17.458887 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:48:17.458887 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:48:17.458887 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:48:17.458887 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:48:17.459732 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:48:17.459732 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Jun 23 15:48:17.459732 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:48:17.459732 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:48:17.459732 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:48:17.459732 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:48:17.459732 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:48:17.459732 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:48:17.459732 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:48:17.459732 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:48:17.459732 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 15:48:17.459732 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 15:48:17.459732 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 23 15:48:17.459732 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Jun 23 15:48:17.459732 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 371/775] 10.222.0.46 () {56 vars in 1111 bytes} [Thu Jun 23 15:48:17 2022] GET /identity/v3/services/network => generated 88 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:48:17.465924 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327999a8-7ca4-43a7-bdd6-8b1ad7df552c None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:17.471835 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:17.567063 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:17.567753 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034c9ef7-a2de-4426-8250-b83ebfd5a7bb None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:17.592210 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327999a8-7ca4-43a7-bdd6-8b1ad7df552c None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:17.610560 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad90f57-8b03-4698-8e39-4155f1fcc524 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:17.611247 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad90f57-8b03-4698-8e39-4155f1fcc524 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:17.611535 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad90f57-8b03-4698-8e39-4155f1fcc524 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:17.611812 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad90f57-8b03-4698-8e39-4155f1fcc524 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:17.612650 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad90f57-8b03-4698-8e39-4155f1fcc524 None admin] RBAC: Adding query filter params (role.id=bfa8ecadc3d5483e821420aa0f0fb41e, scope.project.id=28ecdf5b474c44098c722f677d2a0b77, user.id=be48addfaa154555b8d0cb7aac8b32ac) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:48:17.613198 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad90f57-8b03-4698-8e39-4155f1fcc524 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:17.614438 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad90f57-8b03-4698-8e39-4155f1fcc524 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:17.623115 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 405/776] 10.222.0.46 () {56 vars in 1374 bytes} [Thu Jun 23 15:48:17 2022] GET /identity/v3/role_assignments?user.id=be48addfaa154555b8d0cb7aac8b32ac&scope.project.id=28ecdf5b474c44098c722f677d2a0b77&role.id=bfa8ecadc3d5483e821420aa0f0fb41e => generated 259 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:48:17.624500 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b637451a-4593-4e41-85bd-70e38a1fecf2 None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:17.624797 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b637451a-4593-4e41-85bd-70e38a1fecf2 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:17.625075 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b637451a-4593-4e41-85bd-70e38a1fecf2 None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:17.630777 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 406/777] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jun 23 15:48:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 15:48:17.632612 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd36e1c-58f9-4ca4-be46-b17fa813e510 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:17.632906 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd36e1c-58f9-4ca4-be46-b17fa813e510 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:17.633188 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd36e1c-58f9-4ca4-be46-b17fa813e510 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:17.634281 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb4ddb2-7b41-4872-bb22-148cd344848d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:17.635413 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb4ddb2-7b41-4872-bb22-148cd344848d None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:17.635921 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb4ddb2-7b41-4872-bb22-148cd344848d None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:17.636398 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb4ddb2-7b41-4872-bb22-148cd344848d None admin] PATH_INFO: `/v3/services` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:17.637356 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb4ddb2-7b41-4872-bb22-148cd344848d None admin] RBAC: Adding query filter params (name=network) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:48:17.637863 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb4ddb2-7b41-4872-bb22-148cd344848d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:17.639670 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb4ddb2-7b41-4872-bb22-148cd344848d None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:17.646131 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 372/778] 10.222.0.46 () {56 vars in 1112 bytes} [Thu Jun 23 15:48:17 2022] GET /identity/v3/services?name=network => generated 124 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:48:17.656437 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-5cd36e1c-58f9-4ca4-be46-b17fa813e510 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:48:17.656980 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb4ddb2-7b41-4872-bb22-148cd344848d None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:17.657669 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-5cd36e1c-58f9-4ca4-be46-b17fa813e510 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:48:17.757689 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 407/779] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jun 23 15:48:17 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 23 15:48:17.795394 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb4ddb2-7b41-4872-bb22-148cd344848d None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:17.801520 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd36e1c-58f9-4ca4-be46-b17fa813e510 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:17.842181 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:17.852516 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bf9798-169a-42b9-8aac-d54addbde2de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:17.853473 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bf9798-169a-42b9-8aac-d54addbde2de None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:17.853915 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bf9798-169a-42b9-8aac-d54addbde2de None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:17.854621 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bf9798-169a-42b9-8aac-d54addbde2de None admin] PATH_INFO: `/v3/services` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:17.855554 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bf9798-169a-42b9-8aac-d54addbde2de None admin] RBAC: Adding query filter params (type=network) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:48:17.856040 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bf9798-169a-42b9-8aac-d54addbde2de None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:17.857638 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bf9798-169a-42b9-8aac-d54addbde2de None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:17.872965 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 373/780] 10.222.0.46 () {56 vars in 1112 bytes} [Thu Jun 23 15:48:17 2022] GET /identity/v3/services?type=network => generated 350 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 15:48:17.881748 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bf9798-169a-42b9-8aac-d54addbde2de None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:17.912786 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:17.941389 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd36e1c-58f9-4ca4-be46-b17fa813e510 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:17.985617 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c578c4cf-a589-4d48-9295-9bb2780c9016 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:17.986618 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c578c4cf-a589-4d48-9295-9bb2780c9016 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:17.987028 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c578c4cf-a589-4d48-9295-9bb2780c9016 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:17.987402 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c578c4cf-a589-4d48-9295-9bb2780c9016 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:18.000644 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c578c4cf-a589-4d48-9295-9bb2780c9016 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 15:48:18.009586 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-c578c4cf-a589-4d48-9295-9bb2780c9016 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 15:48:18.009586 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:48:18.009586 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:48:18.009586 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:48:18.009586 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:48:18.009586 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:48:18.009586 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:48:18.009586 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:48:18.009586 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:48:18.009586 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:48:18.009586 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:48:18.009586 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:48:18.009586 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 15:48:18.009586 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 15:48:18.009586 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 15:48:18.009586 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 15:48:18.010756 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 15:48:18.010756 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 15:48:18.010756 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 15:48:18.010756 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 15:48:18.010756 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:48:18.010756 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:48:18.010756 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:48:18.010756 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:48:18.010756 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:48:18.010756 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:48:18.010756 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:48:18.010756 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Jun 23 15:48:18.010756 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:48:18.010756 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:48:18.010756 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:48:18.010756 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:48:18.011800 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:48:18.011800 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:48:18.011800 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:48:18.011800 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:48:18.011800 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 15:48:18.011800 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 15:48:18.011800 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 15:48:18.011800 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 15:48:18.011800 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 15:48:18.011800 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 15:48:18.011800 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 15:48:18.011800 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Jun 23 15:48:18.013872 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 408/781] 10.222.0.46 () {56 vars in 1096 bytes} [Thu Jun 23 15:48:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 15:48:18.019480 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c578c4cf-a589-4d48-9295-9bb2780c9016 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:18.100839 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bf9798-169a-42b9-8aac-d54addbde2de None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:18.123471 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:18.137779 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c578c4cf-a589-4d48-9295-9bb2780c9016 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:18.175441 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3915ac99-7072-444b-892d-af4c887c2cf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:18.176444 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3915ac99-7072-444b-892d-af4c887c2cf9 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:18.176773 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3915ac99-7072-444b-892d-af4c887c2cf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:18.177059 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3915ac99-7072-444b-892d-af4c887c2cf9 None admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:18.177970 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3915ac99-7072-444b-892d-af4c887c2cf9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:48:18.178291 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3915ac99-7072-444b-892d-af4c887c2cf9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:18.179732 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3915ac99-7072-444b-892d-af4c887c2cf9 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:18.190040 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 409/782] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Jun 23 15:48:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:48:18.195795 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3915ac99-7072-444b-892d-af4c887c2cf9 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:18.210027 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44408cb7-a2f3-4c5f-a06e-607bcc5c0f83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:18.210988 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44408cb7-a2f3-4c5f-a06e-607bcc5c0f83 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:18.211438 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44408cb7-a2f3-4c5f-a06e-607bcc5c0f83 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:18.211865 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44408cb7-a2f3-4c5f-a06e-607bcc5c0f83 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:18.212911 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44408cb7-a2f3-4c5f-a06e-607bcc5c0f83 None admin] RBAC: Adding query filter params (interface=public, service_id=c0e1dc9e5f5f452aa7377f8963d251d1, region_id=RegionOne) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:48:18.213469 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44408cb7-a2f3-4c5f-a06e-607bcc5c0f83 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:18.215150 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44408cb7-a2f3-4c5f-a06e-607bcc5c0f83 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:18.234201 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 374/783] 10.222.0.46 () {56 vars in 1251 bytes} [Thu Jun 23 15:48:17 2022] GET /identity/v3/endpoints?service_id=c0e1dc9e5f5f452aa7377f8963d251d1&interface=public®ion_id=RegionOne => generated 194 bytes in 353 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 15:48:18.250067 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44408cb7-a2f3-4c5f-a06e-607bcc5c0f83 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:18.312175 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3915ac99-7072-444b-892d-af4c887c2cf9 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:18.356677 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f4c0e3-47d2-4a92-aa22-fc67f5978324 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:18.357489 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f4c0e3-47d2-4a92-aa22-fc67f5978324 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:18.357776 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f4c0e3-47d2-4a92-aa22-fc67f5978324 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:18.358052 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f4c0e3-47d2-4a92-aa22-fc67f5978324 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:18.379208 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f4c0e3-47d2-4a92-aa22-fc67f5978324 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:18.383331 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f4c0e3-47d2-4a92-aa22-fc67f5978324 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:18.394207 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:18.398680 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:18.410216 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-a5f4c0e3-47d2-4a92-aa22-fc67f5978324 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 15:48:18.410216 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:48:18.410216 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:48:18.410216 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:48:18.410216 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:48:18.410216 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:48:18.410216 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:48:18.410216 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:48:18.410216 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:48:18.410216 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:48:18.410216 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:48:18.410216 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:48:18.410216 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 15:48:18.410216 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 15:48:18.410216 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 15:48:18.411239 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 15:48:18.411239 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:48:18.411239 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:48:18.411239 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:48:18.411239 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:48:18.411239 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:48:18.411239 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:48:18.411239 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:48:18.411239 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Jun 23 15:48:18.411239 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:48:18.411239 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:48:18.411239 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:48:18.411239 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:48:18.411239 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:48:18.411239 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:48:18.411239 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:48:18.411239 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:48:18.412159 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 15:48:18.412159 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 15:48:18.412159 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 15:48:18.412159 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Jun 23 15:48:18.414409 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 410/784] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Jun 23 15:48:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:48:18.418551 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f4c0e3-47d2-4a92-aa22-fc67f5978324 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:18.446612 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44408cb7-a2f3-4c5f-a06e-607bcc5c0f83 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:18.525069 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef0f60c-a5f4-4da5-aaef-1bfb161ff025 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:18.525767 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef0f60c-a5f4-4da5-aaef-1bfb161ff025 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:18.526052 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef0f60c-a5f4-4da5-aaef-1bfb161ff025 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:18.526332 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef0f60c-a5f4-4da5-aaef-1bfb161ff025 None admin] PATH_INFO: `/v3/services` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:18.527658 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef0f60c-a5f4-4da5-aaef-1bfb161ff025 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:18.529042 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef0f60c-a5f4-4da5-aaef-1bfb161ff025 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:18.547033 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 375/785] 10.222.0.46 () {56 vars in 1087 bytes} [Thu Jun 23 15:48:18 2022] GET /identity/v3/services => generated 772 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 15:48:18.601069 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f4c0e3-47d2-4a92-aa22-fc67f5978324 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:18.661698 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b4050f-315b-4ac8-8a96-b7c551e973f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:18.662399 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b4050f-315b-4ac8-8a96-b7c551e973f6 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:18.662701 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b4050f-315b-4ac8-8a96-b7c551e973f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:18.662999 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b4050f-315b-4ac8-8a96-b7c551e973f6 None admin] PATH_INFO: `/v3/domains` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:18.663750 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b4050f-315b-4ac8-8a96-b7c551e973f6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:48:18.664063 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b4050f-315b-4ac8-8a96-b7c551e973f6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:18.665480 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b4050f-315b-4ac8-8a96-b7c551e973f6 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:18.689507 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 411/786] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Jun 23 15:48:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:48:18.695187 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef0f60c-a5f4-4da5-aaef-1bfb161ff025 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:18.813023 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef0f60c-a5f4-4da5-aaef-1bfb161ff025 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:18.849159 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf8624f-d9d7-43ae-ae34-d8a2aaefa351 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:18.849978 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf8624f-d9d7-43ae-ae34-d8a2aaefa351 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:18.850267 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf8624f-d9d7-43ae-ae34-d8a2aaefa351 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:18.850547 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf8624f-d9d7-43ae-ae34-d8a2aaefa351 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:18.858400 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf8624f-d9d7-43ae-ae34-d8a2aaefa351 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:18.859886 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf8624f-d9d7-43ae-ae34-d8a2aaefa351 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:18.861669 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:18.883516 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-4cf8624f-d9d7-43ae-ae34-d8a2aaefa351 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 23 15:48:18.883516 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:48:18.883516 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:48:18.883516 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:48:18.883516 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:48:18.883516 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:48:18.883516 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:48:18.883516 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:48:18.883516 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:48:18.883516 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:48:18.883516 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:48:18.883516 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:48:18.883516 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 15:48:18.883516 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 15:48:18.883516 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 15:48:18.883516 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 15:48:18.884518 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:48:18.884518 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:48:18.884518 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 15:48:18.884518 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 15:48:18.884518 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 15:48:18.884518 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 15:48:18.884518 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:48:18.884518 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:48:18.884518 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:48:18.884518 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:48:18.884518 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:48:18.884518 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Jun 23 15:48:18.884518 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:48:18.884518 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:48:18.884518 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:48:18.884518 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:48:18.884518 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:48:18.884518 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:48:18.885459 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:48:18.885459 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:48:18.885459 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 15:48:18.885459 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 15:48:18.885459 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 15:48:18.885459 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 15:48:18.885459 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 15:48:18.885459 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 15:48:18.885459 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 23 15:48:18.885459 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Jun 23 15:48:18.885459 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 376/787] 10.222.0.46 () {56 vars in 1096 bytes} [Thu Jun 23 15:48:18 2022] GET /identity/v3/users/swift => generated 83 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:48:18.890651 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b4050f-315b-4ac8-8a96-b7c551e973f6 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:18.976867 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:19.024489 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a67b8d6-4470-4d02-a94d-607869aee006 None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:19.025054 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a67b8d6-4470-4d02-a94d-607869aee006 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:19.025322 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a67b8d6-4470-4d02-a94d-607869aee006 None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:19.026720 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 377/788] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jun 23 15:48:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 15:48:19.029518 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b4050f-315b-4ac8-8a96-b7c551e973f6 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:19.032153 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e304254b-6d23-4455-9256-441e0bb7e390 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:19.032509 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e304254b-6d23-4455-9256-441e0bb7e390 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:19.032800 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e304254b-6d23-4455-9256-441e0bb7e390 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:19.064675 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-e304254b-6d23-4455-9256-441e0bb7e390 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:48:19.065791 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-e304254b-6d23-4455-9256-441e0bb7e390 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:48:19.090812 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b6f1ec-c2b7-48b4-95c4-a30762f2ca1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:19.091768 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b6f1ec-c2b7-48b4-95c4-a30762f2ca1f None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:19.092315 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b6f1ec-c2b7-48b4-95c4-a30762f2ca1f None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:19.092757 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b6f1ec-c2b7-48b4-95c4-a30762f2ca1f None admin] PATH_INFO: `/v3/users` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:19.093863 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b6f1ec-c2b7-48b4-95c4-a30762f2ca1f None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:48:19.094319 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b6f1ec-c2b7-48b4-95c4-a30762f2ca1f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:19.096152 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b6f1ec-c2b7-48b4-95c4-a30762f2ca1f None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:19.129778 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 412/789] 10.222.0.46 () {56 vars in 1135 bytes} [Thu Jun 23 15:48:18 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:48:19.138856 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b6f1ec-c2b7-48b4-95c4-a30762f2ca1f None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:19.178919 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 378/790] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jun 23 15:48:19 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 23 15:48:19.218908 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e304254b-6d23-4455-9256-441e0bb7e390 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:19.348041 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e304254b-6d23-4455-9256-441e0bb7e390 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:19.349737 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b6f1ec-c2b7-48b4-95c4-a30762f2ca1f None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:19.380574 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:19.390426 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89dd5596-44f0-453a-8b7b-59366c6eff2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:19.391597 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dd5596-44f0-453a-8b7b-59366c6eff2c None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:19.392201 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dd5596-44f0-453a-8b7b-59366c6eff2c None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:19.392630 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dd5596-44f0-453a-8b7b-59366c6eff2c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:19.400329 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:19.406579 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89dd5596-44f0-453a-8b7b-59366c6eff2c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 15:48:19.416116 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-89dd5596-44f0-453a-8b7b-59366c6eff2c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 15:48:19.416116 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:48:19.416116 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:48:19.416116 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:48:19.416116 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:48:19.416116 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:48:19.416116 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:48:19.416116 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:48:19.416116 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:48:19.416116 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:48:19.416116 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:48:19.416116 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:48:19.416116 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 15:48:19.416116 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 15:48:19.416116 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 15:48:19.416116 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 15:48:19.417220 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 15:48:19.417220 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 15:48:19.417220 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 15:48:19.417220 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 15:48:19.417220 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:48:19.417220 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:48:19.417220 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:48:19.417220 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:48:19.417220 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:48:19.417220 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:48:19.417220 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:48:19.417220 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Jun 23 15:48:19.417220 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:48:19.417220 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:48:19.417220 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:48:19.417220 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:48:19.418064 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:48:19.418064 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:48:19.418064 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:48:19.418064 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:48:19.418064 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 15:48:19.418064 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 15:48:19.418064 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 15:48:19.418064 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 15:48:19.418064 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 15:48:19.418064 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 15:48:19.418064 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 15:48:19.418064 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Jun 23 15:48:19.420960 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 379/791] 10.222.0.46 () {56 vars in 1096 bytes} [Thu Jun 23 15:48:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 15:48:19.426701 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89dd5596-44f0-453a-8b7b-59366c6eff2c None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:19.448780 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322eae59-c88b-4810-b6fa-3735908fd87a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:19.449912 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322eae59-c88b-4810-b6fa-3735908fd87a None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:19.450377 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322eae59-c88b-4810-b6fa-3735908fd87a None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:19.450864 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322eae59-c88b-4810-b6fa-3735908fd87a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:19.472887 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322eae59-c88b-4810-b6fa-3735908fd87a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:19.476255 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322eae59-c88b-4810-b6fa-3735908fd87a None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:19.509164 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-322eae59-c88b-4810-b6fa-3735908fd87a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 15:48:19.509164 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:48:19.509164 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:48:19.509164 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:48:19.509164 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:48:19.509164 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:48:19.509164 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:48:19.509164 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:48:19.509164 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:48:19.509164 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:48:19.509164 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:48:19.509164 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:48:19.509164 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 15:48:19.509164 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 15:48:19.509164 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 15:48:19.510103 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 15:48:19.510103 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:48:19.510103 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:48:19.510103 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:48:19.510103 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:48:19.510103 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:48:19.510103 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:48:19.510103 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:48:19.510103 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Jun 23 15:48:19.510103 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:48:19.510103 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:48:19.510103 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:48:19.510103 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:48:19.510103 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:48:19.510103 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:48:19.510103 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:48:19.510103 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:48:19.511037 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 15:48:19.511037 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 15:48:19.511037 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 15:48:19.511037 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Jun 23 15:48:19.513757 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 413/792] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Jun 23 15:48:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 378 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:48:19.519887 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322eae59-c88b-4810-b6fa-3735908fd87a None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:19.641548 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89dd5596-44f0-453a-8b7b-59366c6eff2c None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:19.685490 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ce6430-f378-4680-9784-b39b56e40d44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:19.686432 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ce6430-f378-4680-9784-b39b56e40d44 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:19.686747 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ce6430-f378-4680-9784-b39b56e40d44 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:19.687035 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ce6430-f378-4680-9784-b39b56e40d44 None admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:19.687859 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ce6430-f378-4680-9784-b39b56e40d44 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:48:19.688172 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ce6430-f378-4680-9784-b39b56e40d44 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:19.689253 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322eae59-c88b-4810-b6fa-3735908fd87a None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:19.689624 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ce6430-f378-4680-9784-b39b56e40d44 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:19.700293 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 380/793] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Jun 23 15:48:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:48:19.702219 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca912e3c-0d47-426f-a696-e0d1b47080a9 None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:19.702517 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca912e3c-0d47-426f-a696-e0d1b47080a9 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:19.702848 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca912e3c-0d47-426f-a696-e0d1b47080a9 None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:19.705393 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 381/794] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jun 23 15:48:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 15:48:19.706132 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca912e3c-0d47-426f-a696-e0d1b47080a9 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:19.733015 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c878466-ef86-4b02-971a-a67348ca7f36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:19.734067 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c878466-ef86-4b02-971a-a67348ca7f36 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:19.734555 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c878466-ef86-4b02-971a-a67348ca7f36 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:19.735154 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c878466-ef86-4b02-971a-a67348ca7f36 None admin] PATH_INFO: `/v3/domains` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:19.736266 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c878466-ef86-4b02-971a-a67348ca7f36 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:48:19.736458 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c878466-ef86-4b02-971a-a67348ca7f36 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:19.737864 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c878466-ef86-4b02-971a-a67348ca7f36 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:19.752552 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 414/795] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Jun 23 15:48:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 15:48:19.757220 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0187d7a4-2db1-451a-bc8e-1bec20ad5120 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:19.757220 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0187d7a4-2db1-451a-bc8e-1bec20ad5120 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:19.757220 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0187d7a4-2db1-451a-bc8e-1bec20ad5120 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:19.775964 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-0187d7a4-2db1-451a-bc8e-1bec20ad5120 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:48:19.776626 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-0187d7a4-2db1-451a-bc8e-1bec20ad5120 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:48:19.788925 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:19.808566 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:19.859868 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca912e3c-0d47-426f-a696-e0d1b47080a9 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:19.872192 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 415/796] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jun 23 15:48:19 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 15:48:19.873164 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0187d7a4-2db1-451a-bc8e-1bec20ad5120 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:19.900183 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe74cb6-20aa-4c0f-a0b8-ce96886f83d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:19.901469 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe74cb6-20aa-4c0f-a0b8-ce96886f83d6 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:19.901960 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe74cb6-20aa-4c0f-a0b8-ce96886f83d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:19.902548 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe74cb6-20aa-4c0f-a0b8-ce96886f83d6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:19.916300 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe74cb6-20aa-4c0f-a0b8-ce96886f83d6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:19.919177 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe74cb6-20aa-4c0f-a0b8-ce96886f83d6 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:19.939402 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-2fe74cb6-20aa-4c0f-a0b8-ce96886f83d6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 15:48:19.939402 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:48:19.939402 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:48:19.939402 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:48:19.939402 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:48:19.939402 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:48:19.939402 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:48:19.939402 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:48:19.939402 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:48:19.939402 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:48:19.939402 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:48:19.939402 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:48:19.939402 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 15:48:19.939402 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 15:48:19.939402 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 15:48:19.940346 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 15:48:19.940346 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:48:19.940346 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:48:19.940346 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:48:19.940346 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:48:19.940346 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:48:19.940346 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:48:19.940346 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:48:19.940346 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Jun 23 15:48:19.940346 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:48:19.940346 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:48:19.940346 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:48:19.940346 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:48:19.940346 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:48:19.940346 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:48:19.940346 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:48:19.940346 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:48:19.941243 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 15:48:19.941243 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 15:48:19.941243 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 15:48:19.941243 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Jun 23 15:48:19.943563 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 382/797] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Jun 23 15:48:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:48:19.944627 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe74cb6-20aa-4c0f-a0b8-ce96886f83d6 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:20.005418 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0187d7a4-2db1-451a-bc8e-1bec20ad5120 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:20.051383 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e4d239-aea1-4376-b1aa-3ac185554d62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:20.052233 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e4d239-aea1-4376-b1aa-3ac185554d62 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:20.052523 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e4d239-aea1-4376-b1aa-3ac185554d62 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:20.052804 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e4d239-aea1-4376-b1aa-3ac185554d62 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:20.062462 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e4d239-aea1-4376-b1aa-3ac185554d62 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:20.064621 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e4d239-aea1-4376-b1aa-3ac185554d62 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:20.070035 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe74cb6-20aa-4c0f-a0b8-ce96886f83d6 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:20.072319 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:20.081906 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-46e4d239-aea1-4376-b1aa-3ac185554d62 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 15:48:20.081906 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:48:20.081906 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:48:20.081906 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:48:20.081906 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:48:20.081906 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:48:20.081906 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:48:20.081906 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:48:20.081906 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:48:20.081906 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:48:20.081906 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:48:20.081906 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:48:20.081906 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 15:48:20.081906 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 15:48:20.081906 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 15:48:20.083474 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 15:48:20.083474 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:48:20.083474 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:48:20.083474 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:48:20.083474 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:48:20.083474 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:48:20.083474 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:48:20.083474 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:48:20.083474 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Jun 23 15:48:20.083474 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:48:20.083474 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:48:20.083474 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:48:20.083474 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:48:20.083474 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:48:20.083474 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:48:20.083474 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:48:20.083474 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:48:20.084375 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 15:48:20.084375 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 15:48:20.084375 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 15:48:20.084375 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 15:48:20.084375 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 15:48:20.084375 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 15:48:20.084375 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 15:48:20.084375 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Jun 23 15:48:20.086775 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 416/798] 10.222.0.46 () {56 vars in 1111 bytes} [Thu Jun 23 15:48:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:48:20.087841 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e4d239-aea1-4376-b1aa-3ac185554d62 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:20.115105 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:20.117087 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45908e2-c4e3-4d20-8be5-80a4b2eff1de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:20.117961 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45908e2-c4e3-4d20-8be5-80a4b2eff1de None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:20.118570 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45908e2-c4e3-4d20-8be5-80a4b2eff1de None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:20.118570 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45908e2-c4e3-4d20-8be5-80a4b2eff1de None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:20.119371 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45908e2-c4e3-4d20-8be5-80a4b2eff1de None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:20.120855 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45908e2-c4e3-4d20-8be5-80a4b2eff1de None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:20.134413 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-d45908e2-c4e3-4d20-8be5-80a4b2eff1de None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 23 15:48:20.134413 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:48:20.134413 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 15:48:20.134413 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 15:48:20.134413 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 15:48:20.134413 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 15:48:20.134413 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 15:48:20.134413 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 15:48:20.134413 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 23 15:48:20.134413 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Jun 23 15:48:20.134413 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 15:48:20.134413 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Jun 23 15:48:20.134413 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:48:20.134413 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:48:20.134413 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:48:20.134413 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:48:20.135527 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:48:20.135527 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:48:20.135527 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:48:20.135527 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:48:20.135527 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:48:20.135527 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:48:20.135527 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:48:20.135527 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 15:48:20.135527 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 15:48:20.135527 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 15:48:20.135527 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 15:48:20.135527 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:48:20.135527 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:48:20.135527 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:48:20.135527 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:48:20.135527 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:48:20.136359 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:48:20.136359 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:48:20.136359 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Jun 23 15:48:20.136359 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:48:20.136359 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:48:20.136359 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:48:20.136359 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:48:20.136359 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:48:20.136359 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:48:20.136359 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:48:20.136359 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:48:20.136359 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 15:48:20.136359 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 15:48:20.136359 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 23 15:48:20.136359 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Jun 23 15:48:20.136359 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 383/799] 10.222.0.46 () {56 vars in 1129 bytes} [Thu Jun 23 15:48:19 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:48:20.137219 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45908e2-c4e3-4d20-8be5-80a4b2eff1de None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:20.208669 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e4d239-aea1-4376-b1aa-3ac185554d62 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:20.247333 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbab30fe-ce77-43bc-beb1-afe74848654d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:20.248243 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbab30fe-ce77-43bc-beb1-afe74848654d None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:20.248243 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbab30fe-ce77-43bc-beb1-afe74848654d None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:20.248620 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbab30fe-ce77-43bc-beb1-afe74848654d None admin] PATH_INFO: `/v3/domains` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:20.249019 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45908e2-c4e3-4d20-8be5-80a4b2eff1de None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:20.249476 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbab30fe-ce77-43bc-beb1-afe74848654d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:48:20.249566 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbab30fe-ce77-43bc-beb1-afe74848654d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:20.251223 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbab30fe-ce77-43bc-beb1-afe74848654d None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:20.267192 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 417/800] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Jun 23 15:48:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:48:20.272550 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbab30fe-ce77-43bc-beb1-afe74848654d None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:20.303415 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e0a63a-731b-4354-951d-515165337b97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:20.304397 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e0a63a-731b-4354-951d-515165337b97 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:20.304814 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e0a63a-731b-4354-951d-515165337b97 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:20.305221 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e0a63a-731b-4354-951d-515165337b97 None admin] PATH_INFO: `/v3/projects` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:20.306198 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e0a63a-731b-4354-951d-515165337b97 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:48:20.306682 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e0a63a-731b-4354-951d-515165337b97 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:20.309257 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e0a63a-731b-4354-951d-515165337b97 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:20.337263 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 384/801] 10.222.0.46 () {56 vars in 1148 bytes} [Thu Jun 23 15:48:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:48:20.337263 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fdfac4-7885-4315-8801-693a41fd22ea None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:20.337263 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fdfac4-7885-4315-8801-693a41fd22ea None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:20.337263 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fdfac4-7885-4315-8801-693a41fd22ea None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:20.337263 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 385/802] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jun 23 15:48:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 15:48:20.337263 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fdfac4-7885-4315-8801-693a41fd22ea None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:20.373949 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:20.378000 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:20.402420 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbab30fe-ce77-43bc-beb1-afe74848654d None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:20.442004 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09112665-d261-4471-8c2a-a4262aa7808f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:20.443499 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09112665-d261-4471-8c2a-a4262aa7808f None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:20.443793 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09112665-d261-4471-8c2a-a4262aa7808f None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:20.444070 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09112665-d261-4471-8c2a-a4262aa7808f None admin] PATH_INFO: `/v3/services` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:20.444995 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09112665-d261-4471-8c2a-a4262aa7808f None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:48:20.445630 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09112665-d261-4471-8c2a-a4262aa7808f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:20.447086 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09112665-d261-4471-8c2a-a4262aa7808f None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:20.459082 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 418/803] 10.222.0.46 () {56 vars in 1124 bytes} [Thu Jun 23 15:48:20 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:48:20.459082 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09112665-d261-4471-8c2a-a4262aa7808f None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:20.468890 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fdfac4-7885-4315-8801-693a41fd22ea None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:20.506136 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847fdb4f-ecc2-41a6-97c5-032cea6bf848 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:20.507145 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847fdb4f-ecc2-41a6-97c5-032cea6bf848 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:20.507436 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847fdb4f-ecc2-41a6-97c5-032cea6bf848 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:20.507739 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847fdb4f-ecc2-41a6-97c5-032cea6bf848 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:20.515409 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847fdb4f-ecc2-41a6-97c5-032cea6bf848 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:20.516872 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847fdb4f-ecc2-41a6-97c5-032cea6bf848 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:20.531925 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-847fdb4f-ecc2-41a6-97c5-032cea6bf848 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 23 15:48:20.531925 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:48:20.531925 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:48:20.531925 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:48:20.531925 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:48:20.531925 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:48:20.531925 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:48:20.531925 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:48:20.531925 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:48:20.531925 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:48:20.531925 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:48:20.531925 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:48:20.531925 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 15:48:20.531925 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 15:48:20.531925 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 15:48:20.531925 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 15:48:20.533076 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:48:20.533076 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:48:20.533076 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 15:48:20.533076 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 15:48:20.533076 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 15:48:20.533076 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 15:48:20.533076 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:48:20.533076 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:48:20.533076 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:48:20.533076 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:48:20.533076 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:48:20.533076 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Jun 23 15:48:20.533076 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:48:20.533076 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:48:20.533076 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:48:20.533076 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:48:20.533076 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:48:20.533076 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:48:20.534058 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:48:20.534058 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:48:20.534058 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 15:48:20.534058 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 15:48:20.534058 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 15:48:20.534058 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 15:48:20.534058 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 15:48:20.534058 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 15:48:20.534058 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 23 15:48:20.534058 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Jun 23 15:48:20.534058 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 386/804] 10.222.0.46 () {56 vars in 1093 bytes} [Thu Jun 23 15:48:20 2022] GET /identity/v3/users/nova => generated 82 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:48:20.534728 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe009fa-df81-402d-a29e-f47578d99b2c None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:20.535027 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe009fa-df81-402d-a29e-f47578d99b2c None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:20.535308 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe009fa-df81-402d-a29e-f47578d99b2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:20.552970 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-3fe009fa-df81-402d-a29e-f47578d99b2c None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:48:20.554181 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-3fe009fa-df81-402d-a29e-f47578d99b2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:48:20.600404 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09112665-d261-4471-8c2a-a4262aa7808f None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:20.638723 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 387/805] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jun 23 15:48:20 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 15:48:20.638723 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe009fa-df81-402d-a29e-f47578d99b2c None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:20.644952 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23829c0-4acf-4149-a6f6-c9897dab700b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:20.645937 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23829c0-4acf-4149-a6f6-c9897dab700b None admin] REQUEST_METHOD: `PUT` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:20.646324 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23829c0-4acf-4149-a6f6-c9897dab700b None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:20.646717 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23829c0-4acf-4149-a6f6-c9897dab700b None admin] PATH_INFO: `/v3/projects/28ecdf5b474c44098c722f677d2a0b77/users/242bf02d14724fe4837fcce8730377f9/roles/812d3c567c52469cb2b35709fbdc857c` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:20.649144 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23829c0-4acf-4149-a6f6-c9897dab700b None admin] RBAC: Authorizing `identity:create_grant(project_id=28ecdf5b474c44098c722f677d2a0b77, user_id=242bf02d14724fe4837fcce8730377f9, role_id=812d3c567c52469cb2b35709fbdc857c)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:20.651663 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23829c0-4acf-4149-a6f6-c9897dab700b None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:20.656877 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:20.673065 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 419/806] 10.222.0.46 () {58 vars in 1439 bytes} [Thu Jun 23 15:48:20 2022] PUT /identity/v3/projects/28ecdf5b474c44098c722f677d2a0b77/users/242bf02d14724fe4837fcce8730377f9/roles/812d3c567c52469cb2b35709fbdc857c => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 15:48:20.673065 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23829c0-4acf-4149-a6f6-c9897dab700b None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:20.680287 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:20.769478 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe009fa-df81-402d-a29e-f47578d99b2c None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:20.809617 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96c64c3-133e-4f3e-95ec-ab9b5fe04fb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:20.810596 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96c64c3-133e-4f3e-95ec-ab9b5fe04fb8 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:20.810914 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96c64c3-133e-4f3e-95ec-ab9b5fe04fb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:20.811216 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96c64c3-133e-4f3e-95ec-ab9b5fe04fb8 None admin] PATH_INFO: `/v3/services` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:20.811942 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96c64c3-133e-4f3e-95ec-ab9b5fe04fb8 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:48:20.812255 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96c64c3-133e-4f3e-95ec-ab9b5fe04fb8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:20.813682 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96c64c3-133e-4f3e-95ec-ab9b5fe04fb8 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:20.824990 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 388/807] 10.222.0.46 () {56 vars in 1124 bytes} [Thu Jun 23 15:48:20 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:48:20.824990 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96c64c3-133e-4f3e-95ec-ab9b5fe04fb8 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:20.836084 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23829c0-4acf-4149-a6f6-c9897dab700b None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:20.882067 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77816996-8c4b-472e-96fa-37efa1fb5382 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:20.882811 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77816996-8c4b-472e-96fa-37efa1fb5382 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:20.883097 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77816996-8c4b-472e-96fa-37efa1fb5382 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:20.883374 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77816996-8c4b-472e-96fa-37efa1fb5382 None admin] PATH_INFO: `/v3/users` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:20.884957 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77816996-8c4b-472e-96fa-37efa1fb5382 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:48:20.884957 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77816996-8c4b-472e-96fa-37efa1fb5382 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:20.885970 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77816996-8c4b-472e-96fa-37efa1fb5382 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:20.916881 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 420/808] 10.222.0.46 () {56 vars in 1133 bytes} [Thu Jun 23 15:48:20 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:48:20.918849 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77816996-8c4b-472e-96fa-37efa1fb5382 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:20.936954 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:20.958571 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96c64c3-133e-4f3e-95ec-ab9b5fe04fb8 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:20.995095 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97601671-45a1-43fb-802d-ec50f58f3173 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:20.995900 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97601671-45a1-43fb-802d-ec50f58f3173 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:20.996183 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97601671-45a1-43fb-802d-ec50f58f3173 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:20.996458 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97601671-45a1-43fb-802d-ec50f58f3173 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:21.004418 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:21.008301 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97601671-45a1-43fb-802d-ec50f58f3173 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 15:48:21.016305 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-97601671-45a1-43fb-802d-ec50f58f3173 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 23 15:48:21.016305 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:48:21.016305 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:48:21.016305 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:48:21.016305 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:48:21.016305 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:48:21.016305 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:48:21.016305 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:48:21.016305 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:48:21.016305 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:48:21.016305 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:48:21.016305 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:48:21.016305 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 15:48:21.016305 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 15:48:21.016305 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 15:48:21.016305 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 15:48:21.017327 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 15:48:21.017327 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 15:48:21.017327 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 15:48:21.017327 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 15:48:21.017327 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:48:21.017327 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:48:21.017327 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:48:21.017327 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:48:21.017327 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:48:21.017327 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:48:21.017327 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:48:21.017327 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Jun 23 15:48:21.017327 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:48:21.017327 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:48:21.017327 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:48:21.017327 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:48:21.018947 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:48:21.018947 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:48:21.018947 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:48:21.018947 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:48:21.018947 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 15:48:21.018947 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 15:48:21.018947 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 15:48:21.018947 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 15:48:21.018947 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 15:48:21.018947 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 15:48:21.018947 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 23 15:48:21.018947 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Jun 23 15:48:21.018947 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 389/809] 10.222.0.46 () {56 vars in 1102 bytes} [Thu Jun 23 15:48:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:48:21.018947 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97601671-45a1-43fb-802d-ec50f58f3173 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:21.049952 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77816996-8c4b-472e-96fa-37efa1fb5382 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:21.097911 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079e016d-8361-410c-b650-e1ff801a1f11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:21.099010 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079e016d-8361-410c-b650-e1ff801a1f11 None admin] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:21.099492 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079e016d-8361-410c-b650-e1ff801a1f11 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:21.099971 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079e016d-8361-410c-b650-e1ff801a1f11 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:21.101278 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079e016d-8361-410c-b650-e1ff801a1f11 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:21.103618 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079e016d-8361-410c-b650-e1ff801a1f11 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:21.141915 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 421/810] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Jun 23 15:48:20 2022] POST /identity/v3/endpoints => generated 346 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 15:48:21.141915 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee711d8-4820-4b20-a784-7aa796f5e816 None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:21.141915 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee711d8-4820-4b20-a784-7aa796f5e816 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:21.141915 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee711d8-4820-4b20-a784-7aa796f5e816 None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:21.141915 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 422/811] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jun 23 15:48:21 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 15:48:21.150807 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee711d8-4820-4b20-a784-7aa796f5e816 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:21.158003 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97601671-45a1-43fb-802d-ec50f58f3173 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:21.217433 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc05454a-7247-4ff4-86e4-9f4e6bea3774 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:21.218218 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc05454a-7247-4ff4-86e4-9f4e6bea3774 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:21.219023 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc05454a-7247-4ff4-86e4-9f4e6bea3774 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:21.219315 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc05454a-7247-4ff4-86e4-9f4e6bea3774 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:21.225412 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:21.295335 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee711d8-4820-4b20-a784-7aa796f5e816 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:21.318634 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:21.321789 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc05454a-7247-4ff4-86e4-9f4e6bea3774 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:21.323965 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc05454a-7247-4ff4-86e4-9f4e6bea3774 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:21.345590 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ce1487-ed7e-449f-b977-2f8f0f3e103c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:21.346269 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ce1487-ed7e-449f-b977-2f8f0f3e103c None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:21.346559 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ce1487-ed7e-449f-b977-2f8f0f3e103c None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:21.346867 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ce1487-ed7e-449f-b977-2f8f0f3e103c None admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:21.347632 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ce1487-ed7e-449f-b977-2f8f0f3e103c None admin] RBAC: Adding query filter params (name=service) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:48:21.347943 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ce1487-ed7e-449f-b977-2f8f0f3e103c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:21.349368 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ce1487-ed7e-449f-b977-2f8f0f3e103c None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:21.358976 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 423/812] 10.222.0.46 () {56 vars in 1103 bytes} [Thu Jun 23 15:48:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:48:21.360063 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60646391-99e6-40c5-9e69-b2caaa8172b4 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:21.360353 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60646391-99e6-40c5-9e69-b2caaa8172b4 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:21.360660 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60646391-99e6-40c5-9e69-b2caaa8172b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:21.382953 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-60646391-99e6-40c5-9e69-b2caaa8172b4 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:48:21.384614 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-60646391-99e6-40c5-9e69-b2caaa8172b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:48:21.437936 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 390/813] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Jun 23 15:48:21 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 23 15:48:21.439302 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc05454a-7247-4ff4-86e4-9f4e6bea3774 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:21.482714 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 424/814] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jun 23 15:48:21 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 15:48:21.482714 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60646391-99e6-40c5-9e69-b2caaa8172b4 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:21.494710 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:21.584373 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc05454a-7247-4ff4-86e4-9f4e6bea3774 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:21.591305 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60646391-99e6-40c5-9e69-b2caaa8172b4 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:21.604544 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:21.639218 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9043cda4-e9cc-4f6e-90be-78318812d62a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:21.639450 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f212f587-6388-44a5-a058-8c87d7c5745d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:21.640207 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f212f587-6388-44a5-a058-8c87d7c5745d None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:21.640497 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f212f587-6388-44a5-a058-8c87d7c5745d None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:21.640774 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f212f587-6388-44a5-a058-8c87d7c5745d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:21.641188 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9043cda4-e9cc-4f6e-90be-78318812d62a None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:21.641672 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9043cda4-e9cc-4f6e-90be-78318812d62a None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:21.642146 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9043cda4-e9cc-4f6e-90be-78318812d62a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:21.652729 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f212f587-6388-44a5-a058-8c87d7c5745d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:21.654808 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f212f587-6388-44a5-a058-8c87d7c5745d None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:21.657915 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9043cda4-e9cc-4f6e-90be-78318812d62a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:21.659928 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9043cda4-e9cc-4f6e-90be-78318812d62a None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:21.677418 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-f212f587-6388-44a5-a058-8c87d7c5745d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 15:48:21.677418 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:48:21.677418 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:48:21.677418 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:48:21.677418 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:48:21.677418 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:48:21.677418 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:48:21.677418 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:48:21.677418 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:48:21.677418 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:48:21.677418 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:48:21.677418 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:48:21.677418 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 15:48:21.677418 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 15:48:21.677418 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 15:48:21.678420 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 15:48:21.678420 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:48:21.678420 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:48:21.678420 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:48:21.678420 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:48:21.678420 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:48:21.678420 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:48:21.678420 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:48:21.678420 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Jun 23 15:48:21.678420 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:48:21.678420 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:48:21.678420 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:48:21.678420 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:48:21.678420 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:48:21.678420 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:48:21.678420 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:48:21.678420 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:48:21.679446 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 15:48:21.679446 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 15:48:21.679446 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 15:48:21.679446 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Jun 23 15:48:21.683021 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 391/815] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Jun 23 15:48:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:48:21.683880 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f212f587-6388-44a5-a058-8c87d7c5745d None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:21.685223 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-9043cda4-e9cc-4f6e-90be-78318812d62a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 15:48:21.685223 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:48:21.685223 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:48:21.685223 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:48:21.685223 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:48:21.685223 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:48:21.685223 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:48:21.685223 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:48:21.685223 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:48:21.685223 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:48:21.685223 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:48:21.685223 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:48:21.685223 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 15:48:21.685223 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 15:48:21.685223 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 15:48:21.686164 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 15:48:21.686164 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:48:21.686164 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:48:21.686164 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:48:21.686164 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:48:21.686164 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:48:21.686164 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:48:21.686164 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:48:21.686164 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Jun 23 15:48:21.686164 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:48:21.686164 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:48:21.686164 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:48:21.686164 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:48:21.686164 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:48:21.686164 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:48:21.686164 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:48:21.686164 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:48:21.687069 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 15:48:21.687069 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 15:48:21.687069 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 15:48:21.687069 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Jun 23 15:48:21.690134 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 425/816] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Jun 23 15:48:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:48:21.691220 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9043cda4-e9cc-4f6e-90be-78318812d62a None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:21.801067 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:21.817255 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9043cda4-e9cc-4f6e-90be-78318812d62a None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:21.818147 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f212f587-6388-44a5-a058-8c87d7c5745d None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:21.855209 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bde11d-f72d-47c9-80f7-26e018016451 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:21.856033 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bde11d-f72d-47c9-80f7-26e018016451 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:21.856508 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bde11d-f72d-47c9-80f7-26e018016451 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:21.856885 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bde11d-f72d-47c9-80f7-26e018016451 None admin] PATH_INFO: `/v3/domains` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:21.857287 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7a67d8-50cc-462f-b3bd-00ed9d0aa0e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:21.857818 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23bde11d-f72d-47c9-80f7-26e018016451 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:48:21.858225 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23bde11d-f72d-47c9-80f7-26e018016451 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:21.858582 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7a67d8-50cc-462f-b3bd-00ed9d0aa0e7 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:21.859110 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7a67d8-50cc-462f-b3bd-00ed9d0aa0e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:21.859536 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7a67d8-50cc-462f-b3bd-00ed9d0aa0e7 None admin] PATH_INFO: `/v3/services/network` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:21.860037 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23bde11d-f72d-47c9-80f7-26e018016451 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:21.860519 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7a67d8-50cc-462f-b3bd-00ed9d0aa0e7 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:21.862343 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7a67d8-50cc-462f-b3bd-00ed9d0aa0e7 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:21.877479 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-4c7a67d8-50cc-462f-b3bd-00ed9d0aa0e7 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 23 15:48:21.877479 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:48:21.877479 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 15:48:21.877479 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 15:48:21.877479 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 15:48:21.877479 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 15:48:21.877479 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 15:48:21.877479 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 15:48:21.877479 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 23 15:48:21.877479 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Jun 23 15:48:21.877479 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 15:48:21.877479 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Jun 23 15:48:21.877479 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:48:21.877479 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:48:21.877479 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:48:21.877479 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:48:21.877479 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:48:21.879006 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:48:21.879006 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:48:21.879006 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:48:21.879006 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:48:21.879006 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:48:21.879006 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:48:21.879006 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 15:48:21.879006 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 15:48:21.879006 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 15:48:21.879006 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 15:48:21.879006 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:48:21.879006 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:48:21.879006 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:48:21.879006 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:48:21.879006 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:48:21.879006 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:48:21.879911 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:48:21.879911 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Jun 23 15:48:21.879911 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:48:21.879911 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:48:21.879911 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:48:21.879911 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:48:21.879911 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:48:21.879911 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:48:21.879911 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:48:21.879911 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:48:21.879911 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 15:48:21.879911 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 15:48:21.879911 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 23 15:48:21.879911 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Jun 23 15:48:21.880673 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 426/817] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Jun 23 15:48:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:48:21.880673 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bde11d-f72d-47c9-80f7-26e018016451 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:21.883233 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 392/818] 10.222.0.46 () {56 vars in 1111 bytes} [Thu Jun 23 15:48:21 2022] GET /identity/v3/services/network => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:48:21.884495 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7a67d8-50cc-462f-b3bd-00ed9d0aa0e7 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:21.920736 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:22.012463 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7a67d8-50cc-462f-b3bd-00ed9d0aa0e7 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:22.029227 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bde11d-f72d-47c9-80f7-26e018016451 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:22.049164 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2a2e04-155b-4e9e-9f74-97964081005f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:22.050318 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2a2e04-155b-4e9e-9f74-97964081005f None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:22.050852 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2a2e04-155b-4e9e-9f74-97964081005f None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:22.051332 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2a2e04-155b-4e9e-9f74-97964081005f None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:22.059339 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2a2e04-155b-4e9e-9f74-97964081005f None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:22.061232 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2a2e04-155b-4e9e-9f74-97964081005f None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:22.066196 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d35381d-9249-449c-b3e4-d0b58b98d907 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:22.066890 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d35381d-9249-449c-b3e4-d0b58b98d907 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:22.067206 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d35381d-9249-449c-b3e4-d0b58b98d907 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:22.067484 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d35381d-9249-449c-b3e4-d0b58b98d907 None admin] PATH_INFO: `/v3/domains` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:22.068204 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d35381d-9249-449c-b3e4-d0b58b98d907 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:48:22.068507 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d35381d-9249-449c-b3e4-d0b58b98d907 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:22.069865 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d35381d-9249-449c-b3e4-d0b58b98d907 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:22.078551 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-2b2a2e04-155b-4e9e-9f74-97964081005f None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 23 15:48:22.078551 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:48:22.078551 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:48:22.078551 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:48:22.078551 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:48:22.078551 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:48:22.078551 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:48:22.078551 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:48:22.078551 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:48:22.078551 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:48:22.078551 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:48:22.078551 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:48:22.078551 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 15:48:22.078551 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 15:48:22.078551 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 15:48:22.078551 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 15:48:22.079896 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:48:22.079896 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:48:22.079896 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 15:48:22.079896 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 15:48:22.079896 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 15:48:22.079896 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 15:48:22.079896 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:48:22.079896 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:48:22.079896 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:48:22.079896 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:48:22.079896 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:48:22.079896 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Jun 23 15:48:22.079896 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:48:22.079896 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:48:22.079896 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:48:22.079896 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:48:22.079896 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:48:22.079896 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:48:22.080907 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:48:22.080907 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:48:22.080907 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 15:48:22.080907 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 15:48:22.080907 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 15:48:22.080907 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 15:48:22.080907 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 15:48:22.080907 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 15:48:22.080907 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 23 15:48:22.080907 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Jun 23 15:48:22.084212 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 427/819] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Jun 23 15:48:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:48:22.085152 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d35381d-9249-449c-b3e4-d0b58b98d907 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:22.086011 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 393/820] 10.222.0.46 () {56 vars in 1117 bytes} [Thu Jun 23 15:48:21 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:48:22.089139 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2a2e04-155b-4e9e-9f74-97964081005f None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:22.106978 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:22.196276 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2a2e04-155b-4e9e-9f74-97964081005f None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:22.214016 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:22.233932 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fc81d2-3d81-491b-8f64-4329e3adb0a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:22.236417 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d35381d-9249-449c-b3e4-d0b58b98d907 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:22.243167 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fc81d2-3d81-491b-8f64-4329e3adb0a5 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:22.243720 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fc81d2-3d81-491b-8f64-4329e3adb0a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:22.244192 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fc81d2-3d81-491b-8f64-4329e3adb0a5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:22.256271 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48fc81d2-3d81-491b-8f64-4329e3adb0a5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:22.258071 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48fc81d2-3d81-491b-8f64-4329e3adb0a5 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:22.288626 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1b42ff-e04d-4013-9aff-c3b9ae3522e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:22.289301 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1b42ff-e04d-4013-9aff-c3b9ae3522e2 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:22.289586 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1b42ff-e04d-4013-9aff-c3b9ae3522e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:22.289912 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1b42ff-e04d-4013-9aff-c3b9ae3522e2 None admin] PATH_INFO: `/v3/services` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:22.290631 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1b42ff-e04d-4013-9aff-c3b9ae3522e2 None admin] RBAC: Adding query filter params (name=network) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:48:22.290961 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1b42ff-e04d-4013-9aff-c3b9ae3522e2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:22.292337 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1b42ff-e04d-4013-9aff-c3b9ae3522e2 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:22.293903 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-48fc81d2-3d81-491b-8f64-4329e3adb0a5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 15:48:22.293903 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:48:22.293903 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:48:22.293903 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:48:22.293903 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:48:22.293903 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:48:22.293903 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:48:22.293903 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:48:22.293903 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:48:22.293903 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:48:22.293903 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:48:22.293903 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:48:22.293903 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 15:48:22.293903 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 15:48:22.293903 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 15:48:22.294782 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 15:48:22.294782 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:48:22.294782 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:48:22.294782 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:48:22.294782 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:48:22.294782 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:48:22.294782 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:48:22.294782 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:48:22.294782 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Jun 23 15:48:22.294782 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:48:22.294782 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:48:22.294782 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:48:22.294782 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:48:22.294782 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:48:22.294782 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:48:22.294782 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:48:22.294782 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:48:22.295684 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 15:48:22.295684 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 15:48:22.295684 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 15:48:22.295684 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 15:48:22.295684 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 15:48:22.295684 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 15:48:22.295684 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 15:48:22.295684 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Jun 23 15:48:22.300363 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 394/821] 10.222.0.46 () {56 vars in 1111 bytes} [Thu Jun 23 15:48:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:48:22.301549 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fc81d2-3d81-491b-8f64-4329e3adb0a5 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:22.307532 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 428/822] 10.222.0.46 () {56 vars in 1112 bytes} [Thu Jun 23 15:48:22 2022] GET /identity/v3/services?name=network => generated 124 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:48:22.308448 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1b42ff-e04d-4013-9aff-c3b9ae3522e2 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:22.428138 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1b42ff-e04d-4013-9aff-c3b9ae3522e2 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:22.459558 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:22.470518 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b31854-9e4e-4f1b-a1ad-1c2170ca2d89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:22.471956 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b31854-9e4e-4f1b-a1ad-1c2170ca2d89 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:22.472761 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b31854-9e4e-4f1b-a1ad-1c2170ca2d89 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:22.474043 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b31854-9e4e-4f1b-a1ad-1c2170ca2d89 None admin] PATH_INFO: `/v3/projects` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:22.475150 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b31854-9e4e-4f1b-a1ad-1c2170ca2d89 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:48:22.476079 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b31854-9e4e-4f1b-a1ad-1c2170ca2d89 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:22.477880 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b31854-9e4e-4f1b-a1ad-1c2170ca2d89 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:22.498087 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fc81d2-3d81-491b-8f64-4329e3adb0a5 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:22.498825 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 429/823] 10.222.0.46 () {56 vars in 1148 bytes} [Thu Jun 23 15:48:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:48:22.499868 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b31854-9e4e-4f1b-a1ad-1c2170ca2d89 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:22.511802 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:22.675162 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b31854-9e4e-4f1b-a1ad-1c2170ca2d89 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:22.702853 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18e89a4-6b58-44c5-a8dc-607f59e71804 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:22.703591 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18e89a4-6b58-44c5-a8dc-607f59e71804 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:22.704260 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18e89a4-6b58-44c5-a8dc-607f59e71804 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:22.704880 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18e89a4-6b58-44c5-a8dc-607f59e71804 None admin] PATH_INFO: `/v3/users` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:22.706202 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18e89a4-6b58-44c5-a8dc-607f59e71804 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:48:22.706585 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18e89a4-6b58-44c5-a8dc-607f59e71804 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:22.708714 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18e89a4-6b58-44c5-a8dc-607f59e71804 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:22.734378 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 395/824] 10.222.0.46 () {56 vars in 1149 bytes} [Thu Jun 23 15:48:22 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 429 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:48:22.734378 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18e89a4-6b58-44c5-a8dc-607f59e71804 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:22.752024 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3443cd-724d-4ecd-8ac1-1e0695e33d4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:22.752989 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3443cd-724d-4ecd-8ac1-1e0695e33d4b None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:22.753448 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3443cd-724d-4ecd-8ac1-1e0695e33d4b None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:22.753936 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3443cd-724d-4ecd-8ac1-1e0695e33d4b None admin] PATH_INFO: `/v3/services` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:22.755523 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3443cd-724d-4ecd-8ac1-1e0695e33d4b None admin] RBAC: Adding query filter params (type=network) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:48:22.756007 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3443cd-724d-4ecd-8ac1-1e0695e33d4b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:22.761228 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3443cd-724d-4ecd-8ac1-1e0695e33d4b None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:22.778950 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 430/825] 10.222.0.46 () {56 vars in 1112 bytes} [Thu Jun 23 15:48:22 2022] GET /identity/v3/services?type=network => generated 350 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:48:22.782288 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3443cd-724d-4ecd-8ac1-1e0695e33d4b None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:22.855680 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18e89a4-6b58-44c5-a8dc-607f59e71804 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:22.882604 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:22.887531 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:22.893992 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5776fb3-c35d-4903-8954-01f39a814ac7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:22.894682 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5776fb3-c35d-4903-8954-01f39a814ac7 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:22.895003 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5776fb3-c35d-4903-8954-01f39a814ac7 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:22.895315 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5776fb3-c35d-4903-8954-01f39a814ac7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:22.896166 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5776fb3-c35d-4903-8954-01f39a814ac7 None admin] RBAC: Adding query filter params (role.id=812d3c567c52469cb2b35709fbdc857c, scope.project.id=28ecdf5b474c44098c722f677d2a0b77, user.id=705115d19bab424d894bea8b1b48a7f0) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:48:22.896480 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5776fb3-c35d-4903-8954-01f39a814ac7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:22.897951 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5776fb3-c35d-4903-8954-01f39a814ac7 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:22.906841 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 396/826] 10.222.0.46 () {56 vars in 1374 bytes} [Thu Jun 23 15:48:22 2022] GET /identity/v3/role_assignments?user.id=705115d19bab424d894bea8b1b48a7f0&scope.project.id=28ecdf5b474c44098c722f677d2a0b77&role.id=812d3c567c52469cb2b35709fbdc857c => generated 608 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:48:22.906841 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5776fb3-c35d-4903-8954-01f39a814ac7 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:23.035351 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5776fb3-c35d-4903-8954-01f39a814ac7 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:23.042246 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3443cd-724d-4ecd-8ac1-1e0695e33d4b None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:23.079951 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d1bbe7-a437-49f8-af05-c043ac25c6a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:23.080635 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d1bbe7-a437-49f8-af05-c043ac25c6a3 None admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:23.080923 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d1bbe7-a437-49f8-af05-c043ac25c6a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:23.081436 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d1bbe7-a437-49f8-af05-c043ac25c6a3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:23.082638 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d1bbe7-a437-49f8-af05-c043ac25c6a3 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:23.084638 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d1bbe7-a437-49f8-af05-c043ac25c6a3 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:23.090938 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34f02f6-e1b3-47a7-ab9d-f5bbc805cd05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:23.092457 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34f02f6-e1b3-47a7-ab9d-f5bbc805cd05 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:23.092457 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34f02f6-e1b3-47a7-ab9d-f5bbc805cd05 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:23.092718 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34f02f6-e1b3-47a7-ab9d-f5bbc805cd05 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:23.108700 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34f02f6-e1b3-47a7-ab9d-f5bbc805cd05 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:23.111517 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34f02f6-e1b3-47a7-ab9d-f5bbc805cd05 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:23.113463 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 397/827] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Jun 23 15:48:22 2022] POST /identity/v3/endpoints => generated 327 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 15:48:23.115788 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676939ba-b5c3-4566-9c9b-281b0d6e4f49 None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:23.116327 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676939ba-b5c3-4566-9c9b-281b0d6e4f49 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:23.116788 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676939ba-b5c3-4566-9c9b-281b0d6e4f49 None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:23.118830 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 398/828] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jun 23 15:48:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 15:48:23.125369 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8272b96e-2e23-4d77-b762-c1908fbab884 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:23.125988 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8272b96e-2e23-4d77-b762-c1908fbab884 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:23.126490 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8272b96e-2e23-4d77-b762-c1908fbab884 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:23.146179 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-8272b96e-2e23-4d77-b762-c1908fbab884 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:48:23.147417 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-8272b96e-2e23-4d77-b762-c1908fbab884 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:48:23.148265 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-e34f02f6-e1b3-47a7-ab9d-f5bbc805cd05 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 15:48:23.148265 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:48:23.148265 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:48:23.148265 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:48:23.148265 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:48:23.148265 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:48:23.148265 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:48:23.148265 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:48:23.148265 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:48:23.148265 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:48:23.148265 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:48:23.148265 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:48:23.148265 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 15:48:23.148265 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 15:48:23.148265 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 15:48:23.149099 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 15:48:23.149099 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:48:23.149099 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:48:23.149099 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:48:23.149099 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:48:23.149099 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:48:23.149099 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:48:23.149099 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:48:23.149099 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Jun 23 15:48:23.149099 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:48:23.149099 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:48:23.149099 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:48:23.149099 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:48:23.149099 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:48:23.149099 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:48:23.149099 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:48:23.149099 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:48:23.150346 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 15:48:23.150346 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 15:48:23.150346 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 15:48:23.150346 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Jun 23 15:48:23.150346 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 431/829] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Jun 23 15:48:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 370 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:48:23.157130 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34f02f6-e1b3-47a7-ab9d-f5bbc805cd05 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:23.215548 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:23.258605 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 399/830] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jun 23 15:48:23 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 23 15:48:23.284161 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8272b96e-2e23-4d77-b762-c1908fbab884 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:23.305146 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34f02f6-e1b3-47a7-ab9d-f5bbc805cd05 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:23.334391 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:23.349873 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c95a18-7493-48ed-a029-e668df38113c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:23.350542 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c95a18-7493-48ed-a029-e668df38113c None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:23.350867 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c95a18-7493-48ed-a029-e668df38113c None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:23.351384 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c95a18-7493-48ed-a029-e668df38113c None admin] PATH_INFO: `/v3/domains` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:23.351855 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c95a18-7493-48ed-a029-e668df38113c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:48:23.352160 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c95a18-7493-48ed-a029-e668df38113c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:23.353547 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c95a18-7493-48ed-a029-e668df38113c None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:23.369450 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 432/831] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Jun 23 15:48:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 15:48:23.371638 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bf97a5-d786-4332-97ee-2e450aa9c666 None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:23.371638 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bf97a5-d786-4332-97ee-2e450aa9c666 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:23.371638 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bf97a5-d786-4332-97ee-2e450aa9c666 None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:23.376588 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 433/832] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jun 23 15:48:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 15:48:23.379757 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bf97a5-d786-4332-97ee-2e450aa9c666 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:23.409341 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8272b96e-2e23-4d77-b762-c1908fbab884 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:23.455778 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0987c5cd-f7a6-41e2-9d00-2af723ee4a2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:23.456700 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0987c5cd-f7a6-41e2-9d00-2af723ee4a2c None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:23.456998 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0987c5cd-f7a6-41e2-9d00-2af723ee4a2c None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:23.457272 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0987c5cd-f7a6-41e2-9d00-2af723ee4a2c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:23.468972 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0987c5cd-f7a6-41e2-9d00-2af723ee4a2c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 15:48:23.477285 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-0987c5cd-f7a6-41e2-9d00-2af723ee4a2c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 15:48:23.477285 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:48:23.477285 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:48:23.477285 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:48:23.477285 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:48:23.477285 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:48:23.477285 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:48:23.477285 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:48:23.477285 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:48:23.477285 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:48:23.477285 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:48:23.477285 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:48:23.477285 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 15:48:23.477285 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 15:48:23.477285 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 15:48:23.477285 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 15:48:23.478268 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 15:48:23.478268 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 15:48:23.478268 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 15:48:23.478268 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 15:48:23.478268 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:48:23.478268 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:48:23.478268 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:48:23.478268 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:48:23.478268 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:48:23.478268 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:48:23.478268 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:48:23.478268 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Jun 23 15:48:23.478268 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:48:23.478268 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:48:23.478268 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:48:23.478268 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:48:23.479168 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:48:23.479168 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:48:23.479168 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:48:23.479168 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:48:23.479168 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 15:48:23.479168 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 15:48:23.479168 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 15:48:23.479168 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 15:48:23.479168 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 15:48:23.479168 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 15:48:23.479168 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 15:48:23.479168 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Jun 23 15:48:23.482882 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 400/833] 10.222.0.46 () {56 vars in 1096 bytes} [Thu Jun 23 15:48:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 15:48:23.485175 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba93710-678c-4bc8-af8c-74301fe91102 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:23.485543 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba93710-678c-4bc8-af8c-74301fe91102 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:23.485832 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba93710-678c-4bc8-af8c-74301fe91102 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:23.503823 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-2ba93710-678c-4bc8-af8c-74301fe91102 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:48:23.504810 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bf97a5-d786-4332-97ee-2e450aa9c666 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:23.505050 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-2ba93710-678c-4bc8-af8c-74301fe91102 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:48:23.534199 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:23.544878 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f3af4e-dc88-4c9e-8540-19dddf2091a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:23.545968 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f3af4e-dc88-4c9e-8540-19dddf2091a0 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:23.545968 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f3af4e-dc88-4c9e-8540-19dddf2091a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:23.546225 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f3af4e-dc88-4c9e-8540-19dddf2091a0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:23.555806 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f3af4e-dc88-4c9e-8540-19dddf2091a0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:23.557427 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f3af4e-dc88-4c9e-8540-19dddf2091a0 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:23.576277 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-77f3af4e-dc88-4c9e-8540-19dddf2091a0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 15:48:23.576277 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:48:23.576277 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:48:23.576277 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:48:23.576277 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:48:23.576277 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:48:23.576277 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:48:23.576277 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:48:23.576277 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:48:23.576277 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:48:23.576277 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:48:23.576277 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:48:23.576277 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 15:48:23.576277 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 15:48:23.576277 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 15:48:23.577412 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 15:48:23.577412 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:48:23.577412 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:48:23.577412 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:48:23.577412 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:48:23.577412 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:48:23.577412 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:48:23.577412 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:48:23.577412 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Jun 23 15:48:23.577412 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:48:23.577412 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:48:23.577412 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:48:23.577412 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:48:23.577412 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:48:23.577412 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:48:23.577412 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:48:23.577412 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:48:23.578481 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 15:48:23.578481 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 15:48:23.578481 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 15:48:23.578481 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 15:48:23.578481 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 15:48:23.578481 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 15:48:23.578481 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 15:48:23.578481 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Jun 23 15:48:23.578481 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 434/834] 10.222.0.46 () {56 vars in 1111 bytes} [Thu Jun 23 15:48:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:48:23.578481 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f3af4e-dc88-4c9e-8540-19dddf2091a0 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:23.602715 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 401/835] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jun 23 15:48:23 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 15:48:23.602715 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba93710-678c-4bc8-af8c-74301fe91102 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:23.638328 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:23.707320 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f3af4e-dc88-4c9e-8540-19dddf2091a0 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:23.714033 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba93710-678c-4bc8-af8c-74301fe91102 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:23.744996 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6a634e-1254-4580-b221-de0297609940 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:23.745768 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6a634e-1254-4580-b221-de0297609940 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:23.746137 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6a634e-1254-4580-b221-de0297609940 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:23.746486 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6a634e-1254-4580-b221-de0297609940 None admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:23.747397 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6a634e-1254-4580-b221-de0297609940 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:48:23.747725 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6a634e-1254-4580-b221-de0297609940 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:23.749178 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6a634e-1254-4580-b221-de0297609940 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:23.749506 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ec88e6-5f27-4e18-86d7-9967cbac99cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:23.750247 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ec88e6-5f27-4e18-86d7-9967cbac99cf None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:23.750607 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ec88e6-5f27-4e18-86d7-9967cbac99cf None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:23.750983 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ec88e6-5f27-4e18-86d7-9967cbac99cf None admin] PATH_INFO: `/v3/projects` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:23.751857 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ec88e6-5f27-4e18-86d7-9967cbac99cf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:48:23.752234 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ec88e6-5f27-4e18-86d7-9967cbac99cf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:23.753727 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ec88e6-5f27-4e18-86d7-9967cbac99cf None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:23.763159 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 435/836] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Jun 23 15:48:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:48:23.763159 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6a634e-1254-4580-b221-de0297609940 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:23.769129 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 402/837] 10.222.0.46 () {56 vars in 1148 bytes} [Thu Jun 23 15:48:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:48:23.771492 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ec88e6-5f27-4e18-86d7-9967cbac99cf None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:23.804461 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:23.870968 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6a634e-1254-4580-b221-de0297609940 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:23.873338 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:23.885236 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ec88e6-5f27-4e18-86d7-9967cbac99cf None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:23.907649 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b6aedd-2cc3-4a33-9c00-c2ef2edcc9b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:23.908456 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b6aedd-2cc3-4a33-9c00-c2ef2edcc9b1 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:23.908737 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b6aedd-2cc3-4a33-9c00-c2ef2edcc9b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:23.909010 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b6aedd-2cc3-4a33-9c00-c2ef2edcc9b1 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:23.909668 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b6aedd-2cc3-4a33-9c00-c2ef2edcc9b1 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:23.911154 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b6aedd-2cc3-4a33-9c00-c2ef2edcc9b1 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:23.919572 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5b73aa-7eec-4e82-96d8-e5e4b6a2b15e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:23.920374 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5b73aa-7eec-4e82-96d8-e5e4b6a2b15e None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:23.920660 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5b73aa-7eec-4e82-96d8-e5e4b6a2b15e None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:23.920936 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5b73aa-7eec-4e82-96d8-e5e4b6a2b15e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:23.922779 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-56b6aedd-2cc3-4a33-9c00-c2ef2edcc9b1 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 23 15:48:23.922779 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:48:23.922779 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 15:48:23.922779 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 15:48:23.922779 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 15:48:23.922779 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 15:48:23.922779 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 15:48:23.922779 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 15:48:23.922779 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 23 15:48:23.922779 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Jun 23 15:48:23.922779 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 15:48:23.922779 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Jun 23 15:48:23.922779 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:48:23.922779 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:48:23.922779 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:48:23.922779 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:48:23.922779 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:48:23.923710 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:48:23.923710 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:48:23.923710 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:48:23.923710 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:48:23.923710 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:48:23.923710 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:48:23.923710 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 15:48:23.923710 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 15:48:23.923710 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 15:48:23.923710 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 15:48:23.923710 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:48:23.923710 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:48:23.923710 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:48:23.923710 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:48:23.923710 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:48:23.923710 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:48:23.924517 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:48:23.924517 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Jun 23 15:48:23.924517 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:48:23.924517 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:48:23.924517 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:48:23.924517 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:48:23.924517 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:48:23.924517 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:48:23.924517 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:48:23.924517 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:48:23.924517 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 15:48:23.924517 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 15:48:23.924517 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 23 15:48:23.924517 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Jun 23 15:48:23.924517 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 436/838] 10.222.0.46 () {56 vars in 1114 bytes} [Thu Jun 23 15:48:23 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:48:23.925322 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b6aedd-2cc3-4a33-9c00-c2ef2edcc9b1 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:23.937113 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5b73aa-7eec-4e82-96d8-e5e4b6a2b15e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:23.938567 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5b73aa-7eec-4e82-96d8-e5e4b6a2b15e None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:23.956089 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-2c5b73aa-7eec-4e82-96d8-e5e4b6a2b15e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 15:48:23.956089 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:48:23.956089 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:48:23.956089 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:48:23.956089 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:48:23.956089 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:48:23.956089 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:48:23.956089 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:48:23.956089 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:48:23.956089 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:48:23.956089 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:48:23.956089 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:48:23.956089 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 15:48:23.956089 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 15:48:23.956089 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 15:48:23.956995 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 15:48:23.956995 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:48:23.956995 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:48:23.956995 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:48:23.956995 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:48:23.956995 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:48:23.956995 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:48:23.956995 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:48:23.956995 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Jun 23 15:48:23.956995 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:48:23.956995 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:48:23.956995 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:48:23.956995 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:48:23.956995 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:48:23.956995 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:48:23.956995 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:48:23.956995 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:48:23.958489 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 15:48:23.958489 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 15:48:23.958489 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 15:48:23.958489 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Jun 23 15:48:23.958489 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 403/839] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Jun 23 15:48:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:48:23.958800 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5b73aa-7eec-4e82-96d8-e5e4b6a2b15e None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:24.038103 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:24.038572 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b6aedd-2cc3-4a33-9c00-c2ef2edcc9b1 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:24.068480 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5b73aa-7eec-4e82-96d8-e5e4b6a2b15e None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:24.073868 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788601ec-b1ee-4439-847c-1ee8cf51e8c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:24.075022 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788601ec-b1ee-4439-847c-1ee8cf51e8c3 None admin] REQUEST_METHOD: `PUT` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:24.075633 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788601ec-b1ee-4439-847c-1ee8cf51e8c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:24.075982 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788601ec-b1ee-4439-847c-1ee8cf51e8c3 None admin] PATH_INFO: `/v3/projects/28ecdf5b474c44098c722f677d2a0b77/users/be48addfaa154555b8d0cb7aac8b32ac/roles/bfa8ecadc3d5483e821420aa0f0fb41e` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:24.092179 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788601ec-b1ee-4439-847c-1ee8cf51e8c3 None admin] RBAC: Authorizing `identity:create_grant(project_id=28ecdf5b474c44098c722f677d2a0b77, user_id=be48addfaa154555b8d0cb7aac8b32ac, role_id=bfa8ecadc3d5483e821420aa0f0fb41e)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:24.093744 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788601ec-b1ee-4439-847c-1ee8cf51e8c3 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:24.118730 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 437/840] 10.222.0.46 () {58 vars in 1439 bytes} [Thu Jun 23 15:48:23 2022] PUT /identity/v3/projects/28ecdf5b474c44098c722f677d2a0b77/users/be48addfaa154555b8d0cb7aac8b32ac/roles/bfa8ecadc3d5483e821420aa0f0fb41e => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 15:48:24.118730 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788601ec-b1ee-4439-847c-1ee8cf51e8c3 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:24.121820 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6388f0-10f7-4b6b-9eac-47300cb71a2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:24.122524 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6388f0-10f7-4b6b-9eac-47300cb71a2f None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:24.122859 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6388f0-10f7-4b6b-9eac-47300cb71a2f None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:24.123210 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6388f0-10f7-4b6b-9eac-47300cb71a2f None admin] PATH_INFO: `/v3/services` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:24.124063 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6388f0-10f7-4b6b-9eac-47300cb71a2f None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:48:24.124349 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6388f0-10f7-4b6b-9eac-47300cb71a2f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:24.125906 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6388f0-10f7-4b6b-9eac-47300cb71a2f None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:24.132741 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 404/841] 10.222.0.46 () {56 vars in 1114 bytes} [Thu Jun 23 15:48:23 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:48:24.140262 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6388f0-10f7-4b6b-9eac-47300cb71a2f None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:24.146997 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:24.265188 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6388f0-10f7-4b6b-9eac-47300cb71a2f None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:24.276224 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788601ec-b1ee-4439-847c-1ee8cf51e8c3 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:24.311578 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30407fd2-9005-4321-b854-435def202194 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:24.312632 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30407fd2-9005-4321-b854-435def202194 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:24.313272 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30407fd2-9005-4321-b854-435def202194 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:24.313561 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30407fd2-9005-4321-b854-435def202194 None admin] PATH_INFO: `/v3/services` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:24.314290 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30407fd2-9005-4321-b854-435def202194 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:48:24.314602 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30407fd2-9005-4321-b854-435def202194 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:24.316275 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30407fd2-9005-4321-b854-435def202194 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:24.322753 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 405/842] 10.222.0.46 () {56 vars in 1114 bytes} [Thu Jun 23 15:48:24 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:48:24.329583 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d823da8-c3d8-4353-8f96-8ffa57054cd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:24.330539 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d823da8-c3d8-4353-8f96-8ffa57054cd5 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:24.331018 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d823da8-c3d8-4353-8f96-8ffa57054cd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:24.331475 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d823da8-c3d8-4353-8f96-8ffa57054cd5 None admin] PATH_INFO: `/v3/domains` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:24.332430 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d823da8-c3d8-4353-8f96-8ffa57054cd5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:48:24.332959 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d823da8-c3d8-4353-8f96-8ffa57054cd5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:24.334573 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d823da8-c3d8-4353-8f96-8ffa57054cd5 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:24.349074 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 438/843] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Jun 23 15:48:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:48:24.354471 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30407fd2-9005-4321-b854-435def202194 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:24.369248 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:24.471569 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30407fd2-9005-4321-b854-435def202194 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:24.511904 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd21a8fd-f135-4b8a-ad21-299c6326ee92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:24.512723 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd21a8fd-f135-4b8a-ad21-299c6326ee92 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:24.513013 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd21a8fd-f135-4b8a-ad21-299c6326ee92 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:24.513294 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd21a8fd-f135-4b8a-ad21-299c6326ee92 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:24.521527 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd21a8fd-f135-4b8a-ad21-299c6326ee92 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:24.522998 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd21a8fd-f135-4b8a-ad21-299c6326ee92 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:24.537912 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-bd21a8fd-f135-4b8a-ad21-299c6326ee92 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 23 15:48:24.537912 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:48:24.537912 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:48:24.537912 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:48:24.537912 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:48:24.537912 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:48:24.537912 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:48:24.537912 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:48:24.537912 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:48:24.537912 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:48:24.537912 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:48:24.537912 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:48:24.537912 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 15:48:24.537912 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 15:48:24.537912 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 15:48:24.537912 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 15:48:24.538942 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:48:24.538942 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:48:24.538942 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 15:48:24.538942 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 15:48:24.538942 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 15:48:24.538942 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 15:48:24.538942 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:48:24.538942 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:48:24.538942 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:48:24.538942 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:48:24.538942 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:48:24.538942 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Jun 23 15:48:24.538942 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:48:24.538942 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:48:24.538942 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:48:24.538942 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:48:24.538942 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:48:24.538942 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:48:24.539905 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:48:24.539905 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:48:24.539905 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 15:48:24.539905 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 15:48:24.539905 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 15:48:24.539905 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 15:48:24.539905 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 15:48:24.539905 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 15:48:24.539905 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 23 15:48:24.539905 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Jun 23 15:48:24.539905 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 406/844] 10.222.0.46 () {56 vars in 1096 bytes} [Thu Jun 23 15:48:24 2022] GET /identity/v3/users/swift => generated 83 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:48:24.546732 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d823da8-c3d8-4353-8f96-8ffa57054cd5 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:24.642364 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:24.665819 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d823da8-c3d8-4353-8f96-8ffa57054cd5 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:24.701306 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa34d782-a0c3-45a1-9641-ce54b58f224c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:24.702074 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa34d782-a0c3-45a1-9641-ce54b58f224c None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:24.702364 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa34d782-a0c3-45a1-9641-ce54b58f224c None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:24.702642 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa34d782-a0c3-45a1-9641-ce54b58f224c None admin] PATH_INFO: `/v3/users` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:24.703517 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa34d782-a0c3-45a1-9641-ce54b58f224c None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:48:24.703827 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa34d782-a0c3-45a1-9641-ce54b58f224c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:24.705269 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa34d782-a0c3-45a1-9641-ce54b58f224c None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:24.724522 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 439/845] 10.222.0.46 () {56 vars in 1135 bytes} [Thu Jun 23 15:48:24 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 15:48:24.729859 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd21a8fd-f135-4b8a-ad21-299c6326ee92 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:24.801982 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:24.840490 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd21a8fd-f135-4b8a-ad21-299c6326ee92 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:24.876860 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdbca38-ed78-4991-8e57-35eacb1bc214 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:24.877527 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdbca38-ed78-4991-8e57-35eacb1bc214 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:24.877815 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdbca38-ed78-4991-8e57-35eacb1bc214 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:24.878097 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdbca38-ed78-4991-8e57-35eacb1bc214 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:24.951888 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdbca38-ed78-4991-8e57-35eacb1bc214 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:24.953745 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdbca38-ed78-4991-8e57-35eacb1bc214 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:25.021617 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:25.059552 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 407/846] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Jun 23 15:48:24 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 23 15:48:25.064806 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa34d782-a0c3-45a1-9641-ce54b58f224c None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:25.171975 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa34d782-a0c3-45a1-9641-ce54b58f224c None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:25.192205 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dd1ebc-af72-4f55-9d13-a9cb027974ea None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:25.192538 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dd1ebc-af72-4f55-9d13-a9cb027974ea None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:25.192772 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dd1ebc-af72-4f55-9d13-a9cb027974ea None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:25.194372 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 408/847] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jun 23 15:48:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 15:48:25.199612 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc5f7d0-2f3e-4280-b699-a148c87356f8 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:25.199899 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc5f7d0-2f3e-4280-b699-a148c87356f8 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:25.200518 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc5f7d0-2f3e-4280-b699-a148c87356f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:25.212327 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7bb042-854a-4286-b2bc-7dd6c15f0fd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:25.213276 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7bb042-854a-4286-b2bc-7dd6c15f0fd5 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:25.213645 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7bb042-854a-4286-b2bc-7dd6c15f0fd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:25.213950 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7bb042-854a-4286-b2bc-7dd6c15f0fd5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:25.218177 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:25.224698 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-bfc5f7d0-2f3e-4280-b699-a148c87356f8 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:48:25.227830 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-bfc5f7d0-2f3e-4280-b699-a148c87356f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:48:25.229408 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7bb042-854a-4286-b2bc-7dd6c15f0fd5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:25.231040 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7bb042-854a-4286-b2bc-7dd6c15f0fd5 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:25.250011 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-ad7bb042-854a-4286-b2bc-7dd6c15f0fd5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 15:48:25.250011 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:48:25.250011 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:48:25.250011 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:48:25.250011 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:48:25.250011 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:48:25.250011 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:48:25.250011 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:48:25.250011 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:48:25.250011 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:48:25.250011 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:48:25.250011 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:48:25.250011 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 15:48:25.250011 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 15:48:25.250011 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 15:48:25.250968 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 15:48:25.250968 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:48:25.250968 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:48:25.250968 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:48:25.250968 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:48:25.250968 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:48:25.250968 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:48:25.250968 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:48:25.250968 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Jun 23 15:48:25.250968 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:48:25.250968 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:48:25.250968 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:48:25.250968 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:48:25.250968 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:48:25.250968 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:48:25.250968 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:48:25.250968 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:48:25.251917 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 15:48:25.251917 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 15:48:25.251917 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 15:48:25.251917 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Jun 23 15:48:25.251917 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 440/848] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Jun 23 15:48:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:48:25.259986 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7bb042-854a-4286-b2bc-7dd6c15f0fd5 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:25.322865 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 409/849] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jun 23 15:48:25 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 15:48:25.339330 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc5f7d0-2f3e-4280-b699-a148c87356f8 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:25.374797 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7bb042-854a-4286-b2bc-7dd6c15f0fd5 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:25.414936 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd289a9-cc8f-4932-b662-2a8832747d21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:25.415800 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd289a9-cc8f-4932-b662-2a8832747d21 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:25.416090 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd289a9-cc8f-4932-b662-2a8832747d21 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:25.416366 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd289a9-cc8f-4932-b662-2a8832747d21 None admin] PATH_INFO: `/v3/domains` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:25.417095 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd289a9-cc8f-4932-b662-2a8832747d21 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:48:25.417405 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd289a9-cc8f-4932-b662-2a8832747d21 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:25.418819 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd289a9-cc8f-4932-b662-2a8832747d21 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:25.431379 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 441/850] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Jun 23 15:48:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:48:25.436348 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd289a9-cc8f-4932-b662-2a8832747d21 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:25.445391 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:25.451861 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc5f7d0-2f3e-4280-b699-a148c87356f8 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:25.488196 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b67f9d-2a7f-4c19-87c5-ce563518be77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:25.489818 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b67f9d-2a7f-4c19-87c5-ce563518be77 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:25.490381 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b67f9d-2a7f-4c19-87c5-ce563518be77 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:25.490895 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b67f9d-2a7f-4c19-87c5-ce563518be77 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:25.491790 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b67f9d-2a7f-4c19-87c5-ce563518be77 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:25.493353 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b67f9d-2a7f-4c19-87c5-ce563518be77 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:25.500673 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:25.508118 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-27b67f9d-2a7f-4c19-87c5-ce563518be77 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 23 15:48:25.508118 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:48:25.508118 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 15:48:25.508118 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 15:48:25.508118 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 15:48:25.508118 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 15:48:25.508118 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 15:48:25.508118 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 15:48:25.508118 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 23 15:48:25.508118 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Jun 23 15:48:25.508118 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 15:48:25.508118 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Jun 23 15:48:25.508118 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:48:25.508118 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:48:25.508118 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:48:25.508118 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:48:25.509057 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:48:25.509057 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:48:25.509057 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:48:25.509057 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:48:25.509057 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:48:25.509057 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:48:25.509057 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:48:25.509057 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 15:48:25.509057 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 15:48:25.509057 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 15:48:25.509057 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 15:48:25.509057 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:48:25.509057 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:48:25.509057 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:48:25.509057 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:48:25.509057 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:48:25.509894 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:48:25.509894 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:48:25.509894 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Jun 23 15:48:25.509894 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:48:25.509894 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:48:25.509894 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:48:25.509894 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:48:25.509894 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:48:25.509894 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:48:25.509894 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:48:25.509894 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:48:25.509894 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 15:48:25.509894 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 15:48:25.509894 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 23 15:48:25.509894 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Jun 23 15:48:25.517614 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 410/851] 10.222.0.46 () {56 vars in 1132 bytes} [Thu Jun 23 15:48:25 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:48:25.518727 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b67f9d-2a7f-4c19-87c5-ce563518be77 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:25.556327 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd289a9-cc8f-4932-b662-2a8832747d21 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:25.595704 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48896bb6-1612-47df-af3a-9d2b1aadb14d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:25.596326 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48896bb6-1612-47df-af3a-9d2b1aadb14d None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:25.596613 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48896bb6-1612-47df-af3a-9d2b1aadb14d None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:25.596886 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48896bb6-1612-47df-af3a-9d2b1aadb14d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:25.604277 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48896bb6-1612-47df-af3a-9d2b1aadb14d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:25.606106 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48896bb6-1612-47df-af3a-9d2b1aadb14d None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:25.621580 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-48896bb6-1612-47df-af3a-9d2b1aadb14d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 15:48:25.621580 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:48:25.621580 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:48:25.621580 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:48:25.621580 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:48:25.621580 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:48:25.621580 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:48:25.621580 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:48:25.621580 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:48:25.621580 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:48:25.621580 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:48:25.621580 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:48:25.621580 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 15:48:25.621580 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 15:48:25.621580 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 15:48:25.622510 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 15:48:25.622510 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:48:25.622510 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:48:25.622510 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:48:25.622510 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:48:25.622510 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:48:25.622510 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:48:25.622510 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:48:25.622510 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Jun 23 15:48:25.622510 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:48:25.622510 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:48:25.622510 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:48:25.622510 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:48:25.622510 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:48:25.622510 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:48:25.622510 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:48:25.622510 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:48:25.623447 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 15:48:25.623447 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 15:48:25.623447 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 15:48:25.623447 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 15:48:25.623447 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 15:48:25.623447 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 15:48:25.623447 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 15:48:25.623447 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Jun 23 15:48:25.627556 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 442/852] 10.222.0.46 () {56 vars in 1111 bytes} [Thu Jun 23 15:48:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 15:48:25.628569 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48896bb6-1612-47df-af3a-9d2b1aadb14d None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:25.635761 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b67f9d-2a7f-4c19-87c5-ce563518be77 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:25.674294 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db79ed74-9260-470d-9bc2-19f0bd94476a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:25.675472 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db79ed74-9260-470d-9bc2-19f0bd94476a None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:25.675954 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db79ed74-9260-470d-9bc2-19f0bd94476a None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:25.676474 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db79ed74-9260-470d-9bc2-19f0bd94476a None admin] PATH_INFO: `/v3/services` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:25.677484 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db79ed74-9260-470d-9bc2-19f0bd94476a None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:48:25.677984 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db79ed74-9260-470d-9bc2-19f0bd94476a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:25.679568 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db79ed74-9260-470d-9bc2-19f0bd94476a None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:25.685246 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 411/853] 10.222.0.46 () {56 vars in 1126 bytes} [Thu Jun 23 15:48:25 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:48:25.694848 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db79ed74-9260-470d-9bc2-19f0bd94476a None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:25.722822 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:25.737751 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48896bb6-1612-47df-af3a-9d2b1aadb14d None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:25.748089 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:25.775266 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319348c1-0ed2-4d2d-8f3e-de2f5f0ae1e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:25.775944 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319348c1-0ed2-4d2d-8f3e-de2f5f0ae1e4 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:25.776233 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319348c1-0ed2-4d2d-8f3e-de2f5f0ae1e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:25.776516 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319348c1-0ed2-4d2d-8f3e-de2f5f0ae1e4 None admin] PATH_INFO: `/v3/projects` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:25.777313 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319348c1-0ed2-4d2d-8f3e-de2f5f0ae1e4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:48:25.777629 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319348c1-0ed2-4d2d-8f3e-de2f5f0ae1e4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:25.779117 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319348c1-0ed2-4d2d-8f3e-de2f5f0ae1e4 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:25.794967 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 443/854] 10.222.0.46 () {56 vars in 1148 bytes} [Thu Jun 23 15:48:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:48:25.797496 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319348c1-0ed2-4d2d-8f3e-de2f5f0ae1e4 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:25.816381 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db79ed74-9260-470d-9bc2-19f0bd94476a None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:25.860407 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2eaaeb-082b-488d-93cb-15fb462eb560 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:25.861334 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2eaaeb-082b-488d-93cb-15fb462eb560 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:25.861616 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2eaaeb-082b-488d-93cb-15fb462eb560 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:25.861891 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2eaaeb-082b-488d-93cb-15fb462eb560 None admin] PATH_INFO: `/v3/services` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:25.862888 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2eaaeb-082b-488d-93cb-15fb462eb560 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:48:25.863606 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2eaaeb-082b-488d-93cb-15fb462eb560 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:25.865853 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2eaaeb-082b-488d-93cb-15fb462eb560 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:25.872508 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 412/855] 10.222.0.46 () {56 vars in 1126 bytes} [Thu Jun 23 15:48:25 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:48:25.916123 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319348c1-0ed2-4d2d-8f3e-de2f5f0ae1e4 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:25.957244 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18eaa836-8bed-47d4-a4e2-2fa465fb80f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:25.958020 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18eaa836-8bed-47d4-a4e2-2fa465fb80f2 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:25.958309 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18eaa836-8bed-47d4-a4e2-2fa465fb80f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:25.958633 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18eaa836-8bed-47d4-a4e2-2fa465fb80f2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:25.959540 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18eaa836-8bed-47d4-a4e2-2fa465fb80f2 None admin] RBAC: Adding query filter params (role.id=812d3c567c52469cb2b35709fbdc857c, scope.project.id=28ecdf5b474c44098c722f677d2a0b77, user.id=242bf02d14724fe4837fcce8730377f9) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:48:25.959856 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18eaa836-8bed-47d4-a4e2-2fa465fb80f2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:25.961356 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18eaa836-8bed-47d4-a4e2-2fa465fb80f2 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:25.968863 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 444/856] 10.222.0.46 () {56 vars in 1374 bytes} [Thu Jun 23 15:48:25 2022] GET /identity/v3/role_assignments?user.id=242bf02d14724fe4837fcce8730377f9&scope.project.id=28ecdf5b474c44098c722f677d2a0b77&role.id=812d3c567c52469cb2b35709fbdc857c => generated 608 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:48:26.286390 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ceb718-5ce6-4ef9-8b16-f80feec9c6bc None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:26.286840 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ceb718-5ce6-4ef9-8b16-f80feec9c6bc None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:26.287049 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ceb718-5ce6-4ef9-8b16-f80feec9c6bc None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:26.290285 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 413/857] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jun 23 15:48:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 15:48:26.294716 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b750c8e7-6194-43d6-bd9e-809ea4209301 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:26.294936 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b750c8e7-6194-43d6-bd9e-809ea4209301 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:26.295151 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b750c8e7-6194-43d6-bd9e-809ea4209301 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:26.313376 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-b750c8e7-6194-43d6-bd9e-809ea4209301 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:48:26.314575 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-b750c8e7-6194-43d6-bd9e-809ea4209301 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:48:26.342254 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:26.391286 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 445/858] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jun 23 15:48:26 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 15:48:26.409169 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ceb718-5ce6-4ef9-8b16-f80feec9c6bc None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:26.503133 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:26.516617 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ceb718-5ce6-4ef9-8b16-f80feec9c6bc None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:26.543095 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7ed63c-38ac-41ad-bbc1-420843932b5d None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:26.543339 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7ed63c-38ac-41ad-bbc1-420843932b5d None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:26.543562 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7ed63c-38ac-41ad-bbc1-420843932b5d None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:26.546257 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 446/859] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jun 23 15:48:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 15:48:26.550332 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90253004-7eec-41f9-a570-a7d9bb86f849 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:26.550623 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90253004-7eec-41f9-a570-a7d9bb86f849 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:26.552388 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382db52d-a11f-4590-838b-24ad7357c61f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:26.553199 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382db52d-a11f-4590-838b-24ad7357c61f None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:26.553477 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382db52d-a11f-4590-838b-24ad7357c61f None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:26.553749 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382db52d-a11f-4590-838b-24ad7357c61f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:26.556375 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90253004-7eec-41f9-a570-a7d9bb86f849 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:26.565603 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382db52d-a11f-4590-838b-24ad7357c61f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 15:48:26.573277 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-382db52d-a11f-4590-838b-24ad7357c61f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 23 15:48:26.573277 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:48:26.573277 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:48:26.573277 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:48:26.573277 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:48:26.573277 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:48:26.573277 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:48:26.573277 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:48:26.573277 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:48:26.573277 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:48:26.573277 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:48:26.573277 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:48:26.573277 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 15:48:26.573277 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 15:48:26.573277 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 15:48:26.573277 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 15:48:26.574223 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 15:48:26.574223 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 15:48:26.574223 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 15:48:26.574223 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 15:48:26.574223 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:48:26.574223 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:48:26.574223 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:48:26.574223 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:48:26.574223 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:48:26.574223 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:48:26.574223 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:48:26.574223 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Jun 23 15:48:26.574223 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:48:26.574223 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:48:26.574223 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:48:26.574223 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:48:26.575107 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:48:26.575107 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:48:26.575107 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:48:26.575107 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:48:26.575107 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 15:48:26.575107 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 15:48:26.575107 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 15:48:26.575107 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 15:48:26.575107 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 15:48:26.575107 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 15:48:26.575107 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 23 15:48:26.575107 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Jun 23 15:48:26.575107 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 414/860] 10.222.0.46 () {56 vars in 1102 bytes} [Thu Jun 23 15:48:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:48:26.580189 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382db52d-a11f-4590-838b-24ad7357c61f None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:26.582415 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-90253004-7eec-41f9-a570-a7d9bb86f849 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:48:26.585278 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-90253004-7eec-41f9-a570-a7d9bb86f849 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:48:26.676125 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 447/861] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jun 23 15:48:26 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 23 15:48:26.693597 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382db52d-a11f-4590-838b-24ad7357c61f None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:26.698837 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90253004-7eec-41f9-a570-a7d9bb86f849 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:26.735082 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19ec2f9-2bbc-4098-8271-76f545ca48fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:26.735759 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19ec2f9-2bbc-4098-8271-76f545ca48fc None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:26.736038 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19ec2f9-2bbc-4098-8271-76f545ca48fc None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:26.736315 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19ec2f9-2bbc-4098-8271-76f545ca48fc None admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:26.737124 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19ec2f9-2bbc-4098-8271-76f545ca48fc None admin] RBAC: Adding query filter params (name=service) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:48:26.737434 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19ec2f9-2bbc-4098-8271-76f545ca48fc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:26.738864 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19ec2f9-2bbc-4098-8271-76f545ca48fc None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:26.752340 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 415/862] 10.222.0.46 () {56 vars in 1103 bytes} [Thu Jun 23 15:48:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 15:48:26.753647 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19ec2f9-2bbc-4098-8271-76f545ca48fc None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:26.768421 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:26.810368 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:26.820386 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90253004-7eec-41f9-a570-a7d9bb86f849 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:26.865400 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5ea4c5-e8ab-4d55-91ff-0a186c2c2b8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:26.866085 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5ea4c5-e8ab-4d55-91ff-0a186c2c2b8b None admin] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:26.866370 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5ea4c5-e8ab-4d55-91ff-0a186c2c2b8b None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:26.866998 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5ea4c5-e8ab-4d55-91ff-0a186c2c2b8b None admin] PATH_INFO: `/v3/services` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:26.868021 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5ea4c5-e8ab-4d55-91ff-0a186c2c2b8b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:26.869410 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5ea4c5-e8ab-4d55-91ff-0a186c2c2b8b None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:26.869741 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19ec2f9-2bbc-4098-8271-76f545ca48fc None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:26.894718 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 448/863] 10.222.0.46 () {60 vars in 1141 bytes} [Thu Jun 23 15:48:26 2022] POST /identity/v3/services => generated 251 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 15:48:26.919813 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccfff8c-435e-4629-987f-bde1c68dec72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:26.920640 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccfff8c-435e-4629-987f-bde1c68dec72 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:26.920926 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccfff8c-435e-4629-987f-bde1c68dec72 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:26.921204 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccfff8c-435e-4629-987f-bde1c68dec72 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:26.933731 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccfff8c-435e-4629-987f-bde1c68dec72 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:26.935242 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccfff8c-435e-4629-987f-bde1c68dec72 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:26.954930 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-dccfff8c-435e-4629-987f-bde1c68dec72 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 15:48:26.954930 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:48:26.954930 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:48:26.954930 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:48:26.954930 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:48:26.954930 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:48:26.954930 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:48:26.954930 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:48:26.954930 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:48:26.954930 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:48:26.954930 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:48:26.954930 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:48:26.954930 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 15:48:26.954930 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 15:48:26.954930 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 15:48:26.955893 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 15:48:26.955893 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:48:26.955893 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:48:26.955893 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:48:26.955893 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:48:26.955893 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:48:26.955893 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:48:26.955893 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:48:26.955893 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Jun 23 15:48:26.955893 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:48:26.955893 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:48:26.955893 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:48:26.955893 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:48:26.955893 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:48:26.955893 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:48:26.955893 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:48:26.955893 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:48:26.957294 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 15:48:26.957294 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 15:48:26.957294 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 15:48:26.957294 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Jun 23 15:48:26.957294 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 416/864] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Jun 23 15:48:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:48:26.964672 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5ea4c5-e8ab-4d55-91ff-0a186c2c2b8b None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:27.086555 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5ea4c5-e8ab-4d55-91ff-0a186c2c2b8b None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:27.123206 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756d060d-a2e1-4774-ba3d-fc63796b188d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:27.124232 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756d060d-a2e1-4774-ba3d-fc63796b188d None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:27.124709 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756d060d-a2e1-4774-ba3d-fc63796b188d None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:27.125176 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756d060d-a2e1-4774-ba3d-fc63796b188d None admin] PATH_INFO: `/v3/domains` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:27.126092 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756d060d-a2e1-4774-ba3d-fc63796b188d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:48:27.126614 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756d060d-a2e1-4774-ba3d-fc63796b188d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:27.128376 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756d060d-a2e1-4774-ba3d-fc63796b188d None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:27.150185 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 449/865] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Jun 23 15:48:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:48:27.151914 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccfff8c-435e-4629-987f-bde1c68dec72 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:27.219473 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:27.264043 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccfff8c-435e-4629-987f-bde1c68dec72 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:27.298974 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78bdb27-f888-4856-87a5-e5d52fc8fad8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:27.299802 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78bdb27-f888-4856-87a5-e5d52fc8fad8 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:27.300089 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78bdb27-f888-4856-87a5-e5d52fc8fad8 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:27.300369 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78bdb27-f888-4856-87a5-e5d52fc8fad8 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:27.308739 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78bdb27-f888-4856-87a5-e5d52fc8fad8 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:27.310192 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78bdb27-f888-4856-87a5-e5d52fc8fad8 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:27.325588 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-d78bdb27-f888-4856-87a5-e5d52fc8fad8 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 23 15:48:27.325588 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:48:27.325588 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:48:27.325588 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:48:27.325588 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:48:27.325588 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:48:27.325588 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:48:27.325588 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:48:27.325588 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:48:27.325588 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:48:27.325588 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:48:27.325588 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:48:27.325588 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 15:48:27.325588 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 15:48:27.325588 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 15:48:27.325588 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 15:48:27.326541 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:48:27.326541 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:48:27.326541 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 15:48:27.326541 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 15:48:27.326541 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 15:48:27.326541 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 15:48:27.326541 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:48:27.326541 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:48:27.326541 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:48:27.326541 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:48:27.326541 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:48:27.326541 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Jun 23 15:48:27.326541 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:48:27.326541 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:48:27.326541 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:48:27.326541 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:48:27.326541 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:48:27.326541 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:48:27.327916 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:48:27.327916 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:48:27.327916 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 15:48:27.327916 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 15:48:27.327916 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 15:48:27.327916 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 15:48:27.327916 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 15:48:27.327916 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 15:48:27.327916 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 23 15:48:27.327916 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Jun 23 15:48:27.327916 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 417/866] 10.222.0.46 () {56 vars in 1117 bytes} [Thu Jun 23 15:48:27 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:48:27.334383 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756d060d-a2e1-4774-ba3d-fc63796b188d None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:27.350217 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:27.445181 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756d060d-a2e1-4774-ba3d-fc63796b188d None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:27.484351 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ba097e-5081-4612-ba3d-9abc4c2ed4d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:27.485035 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ba097e-5081-4612-ba3d-9abc4c2ed4d1 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:27.485321 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ba097e-5081-4612-ba3d-9abc4c2ed4d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:27.485600 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ba097e-5081-4612-ba3d-9abc4c2ed4d1 None admin] PATH_INFO: `/v3/users` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:27.486453 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ba097e-5081-4612-ba3d-9abc4c2ed4d1 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:48:27.486799 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ba097e-5081-4612-ba3d-9abc4c2ed4d1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:27.488251 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ba097e-5081-4612-ba3d-9abc4c2ed4d1 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:27.507699 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 450/867] 10.222.0.46 () {56 vars in 1149 bytes} [Thu Jun 23 15:48:27 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:48:27.515811 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78bdb27-f888-4856-87a5-e5d52fc8fad8 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:27.631719 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78bdb27-f888-4856-87a5-e5d52fc8fad8 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:27.649196 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:27.667335 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfbacd5-5ce5-4646-a619-4db3f082f723 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:27.668000 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfbacd5-5ce5-4646-a619-4db3f082f723 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:27.668286 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfbacd5-5ce5-4646-a619-4db3f082f723 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:27.668563 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfbacd5-5ce5-4646-a619-4db3f082f723 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:27.739529 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cfbacd5-5ce5-4646-a619-4db3f082f723 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:27.741036 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cfbacd5-5ce5-4646-a619-4db3f082f723 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:27.842773 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 418/868] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Jun 23 15:48:27 2022] GET /identity/v3/auth/tokens => generated 1252 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 23 15:48:27.851794 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ba097e-5081-4612-ba3d-9abc4c2ed4d1 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:27.925034 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:27.958459 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ba097e-5081-4612-ba3d-9abc4c2ed4d1 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:27.994653 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b9e557-3d7d-4be1-9b40-ce205ba73d3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:27.995828 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b9e557-3d7d-4be1-9b40-ce205ba73d3d None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:27.996346 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b9e557-3d7d-4be1-9b40-ce205ba73d3d None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:27.996813 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b9e557-3d7d-4be1-9b40-ce205ba73d3d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:28.007506 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b9e557-3d7d-4be1-9b40-ce205ba73d3d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:28.009219 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b9e557-3d7d-4be1-9b40-ce205ba73d3d None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:28.028577 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9346fca4-b344-4416-9948-7e740c2a34d3 None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:28.028789 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-92b9e557-3d7d-4be1-9b40-ce205ba73d3d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 15:48:28.028789 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:48:28.028789 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:48:28.028789 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:48:28.028789 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:48:28.028789 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:48:28.028789 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:48:28.028789 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:48:28.028789 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:48:28.028789 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:48:28.028789 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:48:28.028789 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:48:28.028789 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 15:48:28.028789 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 15:48:28.028789 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 15:48:28.029599 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 15:48:28.029599 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:48:28.029599 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:48:28.029599 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:48:28.029599 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:48:28.029599 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:48:28.029599 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:48:28.029599 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:48:28.029599 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Jun 23 15:48:28.029599 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:48:28.029599 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:48:28.029599 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:48:28.029599 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:48:28.029599 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:48:28.029599 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:48:28.029599 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:48:28.029599 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:48:28.030509 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 15:48:28.030509 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 15:48:28.030509 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 15:48:28.030509 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Jun 23 15:48:28.030773 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9346fca4-b344-4416-9948-7e740c2a34d3 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:28.030773 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9346fca4-b344-4416-9948-7e740c2a34d3 None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:28.030773 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 419/870] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jun 23 15:48:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 15:48:28.033849 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 451/870] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Jun 23 15:48:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:48:28.036725 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9346fca4-b344-4416-9948-7e740c2a34d3 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:28.037489 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd303d6-6cea-41ac-9462-15e1a10e05c7 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:28.037781 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd303d6-6cea-41ac-9462-15e1a10e05c7 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:28.038053 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd303d6-6cea-41ac-9462-15e1a10e05c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:28.055206 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-2cd303d6-6cea-41ac-9462-15e1a10e05c7 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:48:28.056382 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-2cd303d6-6cea-41ac-9462-15e1a10e05c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:48:28.057873 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:28.138516 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 452/871] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jun 23 15:48:28 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 23 15:48:28.139888 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56117f6d-0f7a-4bfb-8429-8e31531e2da9 None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:28.140179 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56117f6d-0f7a-4bfb-8429-8e31531e2da9 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:28.140461 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56117f6d-0f7a-4bfb-8429-8e31531e2da9 None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:28.141722 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 453/872] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jun 23 15:48:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 15:48:28.147491 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfeefd7-91db-4b42-8d1a-1f4a95284ee7 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:28.147777 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfeefd7-91db-4b42-8d1a-1f4a95284ee7 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:28.148057 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfeefd7-91db-4b42-8d1a-1f4a95284ee7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:28.157516 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9346fca4-b344-4416-9948-7e740c2a34d3 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:28.166935 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-dcfeefd7-91db-4b42-8d1a-1f4a95284ee7 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:48:28.168508 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-dcfeefd7-91db-4b42-8d1a-1f4a95284ee7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:48:28.197070 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e864b9-2871-4220-82a3-43c106ab7d25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:28.197665 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e864b9-2871-4220-82a3-43c106ab7d25 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:28.197946 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e864b9-2871-4220-82a3-43c106ab7d25 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:28.198465 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e864b9-2871-4220-82a3-43c106ab7d25 None admin] PATH_INFO: `/v3/domains` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:28.199033 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e864b9-2871-4220-82a3-43c106ab7d25 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:48:28.199402 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e864b9-2871-4220-82a3-43c106ab7d25 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:28.200758 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e864b9-2871-4220-82a3-43c106ab7d25 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:28.214738 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 420/873] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Jun 23 15:48:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:48:28.215912 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e864b9-2871-4220-82a3-43c106ab7d25 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:28.229759 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:28.252612 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 454/874] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jun 23 15:48:28 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 23 15:48:28.253566 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfeefd7-91db-4b42-8d1a-1f4a95284ee7 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:28.309351 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:28.332082 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e864b9-2871-4220-82a3-43c106ab7d25 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:28.369749 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f39570-73c5-4bf0-a8cc-4592fd9f2c30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:28.370424 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f39570-73c5-4bf0-a8cc-4592fd9f2c30 None admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:28.370733 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f39570-73c5-4bf0-a8cc-4592fd9f2c30 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:28.371022 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f39570-73c5-4bf0-a8cc-4592fd9f2c30 None admin] PATH_INFO: `/v3/services` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:28.371783 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f39570-73c5-4bf0-a8cc-4592fd9f2c30 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:28.373157 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f39570-73c5-4bf0-a8cc-4592fd9f2c30 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:28.374064 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfeefd7-91db-4b42-8d1a-1f4a95284ee7 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:28.388482 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 421/875] 10.222.0.46 () {60 vars in 1141 bytes} [Thu Jun 23 15:48:28 2022] POST /identity/v3/services => generated 269 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 15:48:28.389194 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f39570-73c5-4bf0-a8cc-4592fd9f2c30 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:28.443903 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03db5ed-be54-431d-8f02-45f993ae3b0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:28.445142 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03db5ed-be54-431d-8f02-45f993ae3b0b None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:28.445625 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03db5ed-be54-431d-8f02-45f993ae3b0b None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:28.447207 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03db5ed-be54-431d-8f02-45f993ae3b0b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:28.461247 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03db5ed-be54-431d-8f02-45f993ae3b0b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:28.463194 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03db5ed-be54-431d-8f02-45f993ae3b0b None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:28.485376 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-b03db5ed-be54-431d-8f02-45f993ae3b0b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 15:48:28.485376 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:48:28.485376 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:48:28.485376 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:48:28.485376 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:48:28.485376 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:48:28.485376 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:48:28.485376 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:48:28.485376 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:48:28.485376 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:48:28.485376 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:48:28.485376 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:48:28.485376 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 15:48:28.485376 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 15:48:28.485376 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 15:48:28.486325 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 15:48:28.486325 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:48:28.486325 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:48:28.486325 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:48:28.486325 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:48:28.486325 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:48:28.486325 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:48:28.486325 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:48:28.486325 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Jun 23 15:48:28.486325 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:48:28.486325 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:48:28.486325 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:48:28.486325 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:48:28.486325 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:48:28.486325 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:48:28.486325 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:48:28.486325 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:48:28.487324 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 15:48:28.487324 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 15:48:28.487324 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 15:48:28.487324 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 15:48:28.487324 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 15:48:28.487324 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 15:48:28.487324 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 15:48:28.487324 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Jun 23 15:48:28.490010 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 455/876] 10.222.0.46 () {56 vars in 1111 bytes} [Thu Jun 23 15:48:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:48:28.495470 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03db5ed-be54-431d-8f02-45f993ae3b0b None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:28.532361 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f39570-73c5-4bf0-a8cc-4592fd9f2c30 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:28.563507 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:28.585144 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9324204a-66f7-41eb-83dc-c9b830ec8858 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:28.585979 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9324204a-66f7-41eb-83dc-c9b830ec8858 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:28.586261 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9324204a-66f7-41eb-83dc-c9b830ec8858 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:28.586533 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9324204a-66f7-41eb-83dc-c9b830ec8858 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:28.587238 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9324204a-66f7-41eb-83dc-c9b830ec8858 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:28.588603 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9324204a-66f7-41eb-83dc-c9b830ec8858 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:28.603963 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-9324204a-66f7-41eb-83dc-c9b830ec8858 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 23 15:48:28.603963 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:48:28.603963 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 15:48:28.603963 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 15:48:28.603963 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 15:48:28.603963 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 15:48:28.603963 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 15:48:28.603963 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 15:48:28.603963 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 23 15:48:28.603963 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Jun 23 15:48:28.603963 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 15:48:28.603963 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Jun 23 15:48:28.603963 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:48:28.603963 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:48:28.603963 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:48:28.603963 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:48:28.603963 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:48:28.604995 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:48:28.604995 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:48:28.604995 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:48:28.604995 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:48:28.604995 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:48:28.604995 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:48:28.604995 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 15:48:28.604995 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 15:48:28.604995 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 15:48:28.604995 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 15:48:28.604995 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:48:28.604995 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:48:28.604995 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:48:28.604995 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:48:28.604995 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:48:28.604995 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:48:28.605832 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:48:28.605832 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Jun 23 15:48:28.605832 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:48:28.605832 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:48:28.605832 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:48:28.605832 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:48:28.605832 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:48:28.605832 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:48:28.605832 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:48:28.605832 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:48:28.605832 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 15:48:28.605832 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 15:48:28.605832 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 23 15:48:28.605832 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Jun 23 15:48:28.605832 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 422/877] 10.222.0.46 () {56 vars in 1126 bytes} [Thu Jun 23 15:48:28 2022] GET /identity/v3/services/object-store => generated 93 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:48:28.611201 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9324204a-66f7-41eb-83dc-c9b830ec8858 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:28.614656 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:28.646261 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03db5ed-be54-431d-8f02-45f993ae3b0b None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:28.690137 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f1aae9-aa63-4243-a6f0-54cadc6a55bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:28.690989 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f1aae9-aa63-4243-a6f0-54cadc6a55bc None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:28.691372 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f1aae9-aa63-4243-a6f0-54cadc6a55bc None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:28.692437 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f1aae9-aa63-4243-a6f0-54cadc6a55bc None admin] PATH_INFO: `/v3/projects` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:28.693829 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f1aae9-aa63-4243-a6f0-54cadc6a55bc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:48:28.694249 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f1aae9-aa63-4243-a6f0-54cadc6a55bc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:28.696274 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f1aae9-aa63-4243-a6f0-54cadc6a55bc None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:28.716944 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 456/878] 10.222.0.46 () {56 vars in 1148 bytes} [Thu Jun 23 15:48:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:48:28.722770 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f1aae9-aa63-4243-a6f0-54cadc6a55bc None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:28.739239 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9324204a-66f7-41eb-83dc-c9b830ec8858 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:28.782113 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf56aa9-abbb-4a6e-b11d-6397ed39eac8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:28.783118 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf56aa9-abbb-4a6e-b11d-6397ed39eac8 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:28.783946 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf56aa9-abbb-4a6e-b11d-6397ed39eac8 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:28.784378 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf56aa9-abbb-4a6e-b11d-6397ed39eac8 None admin] PATH_INFO: `/v3/services` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:28.785610 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf56aa9-abbb-4a6e-b11d-6397ed39eac8 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:48:28.786414 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf56aa9-abbb-4a6e-b11d-6397ed39eac8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:28.788240 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf56aa9-abbb-4a6e-b11d-6397ed39eac8 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:28.795471 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 423/879] 10.222.0.46 () {56 vars in 1122 bytes} [Thu Jun 23 15:48:28 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 15:48:28.800307 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf56aa9-abbb-4a6e-b11d-6397ed39eac8 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:28.852470 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:28.858701 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f1aae9-aa63-4243-a6f0-54cadc6a55bc None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:28.893848 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:28.910315 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24260fa1-0342-4435-85ed-8b0225039557 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:28.911573 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24260fa1-0342-4435-85ed-8b0225039557 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:28.912031 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24260fa1-0342-4435-85ed-8b0225039557 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:28.912759 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24260fa1-0342-4435-85ed-8b0225039557 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:28.914027 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24260fa1-0342-4435-85ed-8b0225039557 None admin] RBAC: Adding query filter params (role.id=bfa8ecadc3d5483e821420aa0f0fb41e, scope.project.id=28ecdf5b474c44098c722f677d2a0b77, user.id=be48addfaa154555b8d0cb7aac8b32ac) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:48:28.914485 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24260fa1-0342-4435-85ed-8b0225039557 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:28.916683 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24260fa1-0342-4435-85ed-8b0225039557 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:28.924718 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 457/880] 10.222.0.46 () {56 vars in 1374 bytes} [Thu Jun 23 15:48:28 2022] GET /identity/v3/role_assignments?user.id=be48addfaa154555b8d0cb7aac8b32ac&scope.project.id=28ecdf5b474c44098c722f677d2a0b77&role.id=bfa8ecadc3d5483e821420aa0f0fb41e => generated 608 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:48:28.944578 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf56aa9-abbb-4a6e-b11d-6397ed39eac8 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:28.983508 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47c6fa3-eff8-429c-88a1-c756193992bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:28.984124 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47c6fa3-eff8-429c-88a1-c756193992bb None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:28.984409 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47c6fa3-eff8-429c-88a1-c756193992bb None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:28.984687 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47c6fa3-eff8-429c-88a1-c756193992bb None admin] PATH_INFO: `/v3/services` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:28.985420 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47c6fa3-eff8-429c-88a1-c756193992bb None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:48:28.985729 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47c6fa3-eff8-429c-88a1-c756193992bb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:28.987160 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47c6fa3-eff8-429c-88a1-c756193992bb None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:28.993710 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 424/881] 10.222.0.46 () {56 vars in 1122 bytes} [Thu Jun 23 15:48:28 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 15:48:29.205130 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff33ba02-cd05-4b52-b42c-af8614235145 None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:29.205671 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff33ba02-cd05-4b52-b42c-af8614235145 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:29.206091 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff33ba02-cd05-4b52-b42c-af8614235145 None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:29.208356 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 458/882] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jun 23 15:48:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 15:48:29.213839 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41741aed-e126-47ae-9507-146daeafffc5 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:29.214128 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41741aed-e126-47ae-9507-146daeafffc5 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:29.214411 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41741aed-e126-47ae-9507-146daeafffc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:29.236338 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-41741aed-e126-47ae-9507-146daeafffc5 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:48:29.240641 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-41741aed-e126-47ae-9507-146daeafffc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:48:29.310524 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 425/883] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jun 23 15:48:29 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 15:48:29.334402 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff33ba02-cd05-4b52-b42c-af8614235145 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:29.440214 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff33ba02-cd05-4b52-b42c-af8614235145 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:29.476617 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b98e4ed-457c-4cb5-8e7f-1deb1088d818 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:29.477429 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b98e4ed-457c-4cb5-8e7f-1deb1088d818 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:29.477714 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b98e4ed-457c-4cb5-8e7f-1deb1088d818 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:29.477991 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b98e4ed-457c-4cb5-8e7f-1deb1088d818 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:29.478651 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b98e4ed-457c-4cb5-8e7f-1deb1088d818 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:29.480055 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b98e4ed-457c-4cb5-8e7f-1deb1088d818 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:29.492955 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-8b98e4ed-457c-4cb5-8e7f-1deb1088d818 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 23 15:48:29.492955 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:48:29.492955 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 15:48:29.492955 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 15:48:29.492955 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 15:48:29.492955 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 15:48:29.492955 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 15:48:29.492955 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 15:48:29.492955 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 23 15:48:29.492955 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Jun 23 15:48:29.492955 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 15:48:29.492955 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Jun 23 15:48:29.492955 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:48:29.492955 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:48:29.492955 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:48:29.492955 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:48:29.492955 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:48:29.494008 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:48:29.494008 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:48:29.494008 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:48:29.494008 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:48:29.494008 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:48:29.494008 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:48:29.494008 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 15:48:29.494008 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 15:48:29.494008 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 15:48:29.494008 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 15:48:29.494008 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:48:29.494008 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:48:29.494008 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:48:29.494008 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:48:29.494008 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:48:29.494008 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:48:29.495379 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:48:29.495379 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Jun 23 15:48:29.495379 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:48:29.495379 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:48:29.495379 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:48:29.495379 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:48:29.495379 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:48:29.495379 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:48:29.495379 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:48:29.495379 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:48:29.495379 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 15:48:29.495379 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 15:48:29.495379 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 23 15:48:29.495379 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Jun 23 15:48:29.495379 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 459/884] 10.222.0.46 () {56 vars in 1114 bytes} [Thu Jun 23 15:48:29 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:48:29.500328 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41741aed-e126-47ae-9507-146daeafffc5 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:29.568946 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:29.619681 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41741aed-e126-47ae-9507-146daeafffc5 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:29.655800 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7d5504-8a59-402e-8cf0-216f08cacf7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:29.656506 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7d5504-8a59-402e-8cf0-216f08cacf7b None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:29.656793 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7d5504-8a59-402e-8cf0-216f08cacf7b None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:29.657072 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7d5504-8a59-402e-8cf0-216f08cacf7b None admin] PATH_INFO: `/v3/services` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:29.657805 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7d5504-8a59-402e-8cf0-216f08cacf7b None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:48:29.658114 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7d5504-8a59-402e-8cf0-216f08cacf7b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:29.659563 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7d5504-8a59-402e-8cf0-216f08cacf7b None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:29.666275 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 426/885] 10.222.0.46 () {56 vars in 1114 bytes} [Thu Jun 23 15:48:29 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:48:29.672237 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b98e4ed-457c-4cb5-8e7f-1deb1088d818 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:29.703767 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:29.786358 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b98e4ed-457c-4cb5-8e7f-1deb1088d818 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:29.820523 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b60ae7-3ea7-415e-8a19-3a5c79a5e420 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:29.821197 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b60ae7-3ea7-415e-8a19-3a5c79a5e420 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:29.821483 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b60ae7-3ea7-415e-8a19-3a5c79a5e420 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:29.821757 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b60ae7-3ea7-415e-8a19-3a5c79a5e420 None admin] PATH_INFO: `/v3/services` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:29.822523 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b60ae7-3ea7-415e-8a19-3a5c79a5e420 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:48:29.822861 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b60ae7-3ea7-415e-8a19-3a5c79a5e420 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:29.824293 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b60ae7-3ea7-415e-8a19-3a5c79a5e420 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:29.829536 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 460/886] 10.222.0.46 () {56 vars in 1114 bytes} [Thu Jun 23 15:48:29 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:48:29.835886 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7d5504-8a59-402e-8cf0-216f08cacf7b None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:29.941610 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7d5504-8a59-402e-8cf0-216f08cacf7b None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:29.966558 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:29.978736 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e2533d-a7ed-4828-b5fd-19b900acd9a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:29.979416 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e2533d-a7ed-4828-b5fd-19b900acd9a4 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:29.979706 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e2533d-a7ed-4828-b5fd-19b900acd9a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:29.979984 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e2533d-a7ed-4828-b5fd-19b900acd9a4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:29.980783 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e2533d-a7ed-4828-b5fd-19b900acd9a4 None admin] RBAC: Adding query filter params (interface=public, service_id=13623cb1d2a2451ba95554a4e6ee8d08, region_id=RegionOne) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:48:29.981099 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e2533d-a7ed-4828-b5fd-19b900acd9a4 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:29.982466 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e2533d-a7ed-4828-b5fd-19b900acd9a4 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:29.988370 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 427/887] 10.222.0.46 () {56 vars in 1251 bytes} [Thu Jun 23 15:48:29 2022] GET /identity/v3/endpoints?service_id=13623cb1d2a2451ba95554a4e6ee8d08&interface=public®ion_id=RegionOne => generated 194 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:48:29.994124 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b60ae7-3ea7-415e-8a19-3a5c79a5e420 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:30.087785 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:30.097536 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b60ae7-3ea7-415e-8a19-3a5c79a5e420 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:30.133321 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abe8075-de5f-4915-bf53-0634adfc18e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:30.134012 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abe8075-de5f-4915-bf53-0634adfc18e6 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:30.134296 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abe8075-de5f-4915-bf53-0634adfc18e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:30.134570 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abe8075-de5f-4915-bf53-0634adfc18e6 None admin] PATH_INFO: `/v3/services` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:30.135361 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abe8075-de5f-4915-bf53-0634adfc18e6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:30.136785 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abe8075-de5f-4915-bf53-0634adfc18e6 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:30.143814 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 461/888] 10.222.0.46 () {56 vars in 1087 bytes} [Thu Jun 23 15:48:29 2022] GET /identity/v3/services => generated 1268 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 15:48:30.639730 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7d2c68-00bb-42f4-939e-60f2192f20a6 None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:30.640701 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7d2c68-00bb-42f4-939e-60f2192f20a6 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:30.641194 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7d2c68-00bb-42f4-939e-60f2192f20a6 None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:30.643165 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 428/889] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jun 23 15:48:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 15:48:30.648259 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467c8067-9f03-49d6-ac7c-93067f438b00 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:30.648675 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467c8067-9f03-49d6-ac7c-93067f438b00 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:30.648876 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467c8067-9f03-49d6-ac7c-93067f438b00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:30.666541 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-467c8067-9f03-49d6-ac7c-93067f438b00 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:48:30.667790 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-467c8067-9f03-49d6-ac7c-93067f438b00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:48:30.736825 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 462/890] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jun 23 15:48:30 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 15:48:30.761249 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7d2c68-00bb-42f4-939e-60f2192f20a6 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:30.871966 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7d2c68-00bb-42f4-939e-60f2192f20a6 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:30.908118 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81399027-fac6-4298-a4a5-da3ac801a918 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:30.908947 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81399027-fac6-4298-a4a5-da3ac801a918 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:30.909232 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81399027-fac6-4298-a4a5-da3ac801a918 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:30.909509 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81399027-fac6-4298-a4a5-da3ac801a918 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:30.910167 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81399027-fac6-4298-a4a5-da3ac801a918 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:30.912109 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81399027-fac6-4298-a4a5-da3ac801a918 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:30.924763 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-81399027-fac6-4298-a4a5-da3ac801a918 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 23 15:48:30.924763 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:48:30.924763 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 15:48:30.924763 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 15:48:30.924763 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 15:48:30.924763 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 15:48:30.924763 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 15:48:30.924763 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 15:48:30.924763 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 23 15:48:30.924763 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Jun 23 15:48:30.924763 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 15:48:30.924763 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Jun 23 15:48:30.924763 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:48:30.924763 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:48:30.924763 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:48:30.924763 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:48:30.925699 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:48:30.925699 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:48:30.925699 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:48:30.925699 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:48:30.925699 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:48:30.925699 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:48:30.925699 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:48:30.925699 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 15:48:30.925699 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 15:48:30.925699 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 15:48:30.925699 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 15:48:30.925699 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:48:30.925699 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:48:30.925699 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:48:30.925699 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:48:30.925699 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:48:30.927553 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:48:30.927553 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:48:30.927553 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Jun 23 15:48:30.927553 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:48:30.927553 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:48:30.927553 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:48:30.927553 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:48:30.927553 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:48:30.927553 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:48:30.927553 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:48:30.927553 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:48:30.927553 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 15:48:30.927553 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 15:48:30.927553 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 23 15:48:30.927553 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Jun 23 15:48:30.927553 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 429/891] 10.222.0.46 () {56 vars in 1132 bytes} [Thu Jun 23 15:48:30 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:48:30.932705 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467c8067-9f03-49d6-ac7c-93067f438b00 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:31.017426 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:31.034403 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467c8067-9f03-49d6-ac7c-93067f438b00 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:31.069346 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cf542d-7c16-46b5-885c-1e9bb1a83b65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:31.070024 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cf542d-7c16-46b5-885c-1e9bb1a83b65 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:31.070309 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cf542d-7c16-46b5-885c-1e9bb1a83b65 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:31.070589 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cf542d-7c16-46b5-885c-1e9bb1a83b65 None admin] PATH_INFO: `/v3/services` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:31.071344 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89cf542d-7c16-46b5-885c-1e9bb1a83b65 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:48:31.071655 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89cf542d-7c16-46b5-885c-1e9bb1a83b65 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:31.073030 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89cf542d-7c16-46b5-885c-1e9bb1a83b65 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:31.078088 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 463/892] 10.222.0.46 () {56 vars in 1126 bytes} [Thu Jun 23 15:48:30 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:48:31.084205 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81399027-fac6-4298-a4a5-da3ac801a918 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:31.131739 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad619201-c88f-4124-84a1-0c695d7c565f None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:31.132033 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad619201-c88f-4124-84a1-0c695d7c565f None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:31.132197 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad619201-c88f-4124-84a1-0c695d7c565f None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:31.133827 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 464/893] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jun 23 15:48:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 15:48:31.138573 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffb59ab-42a3-4fb7-a3ca-2c6fef9c7f54 None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:31.138898 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffb59ab-42a3-4fb7-a3ca-2c6fef9c7f54 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:31.139179 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffb59ab-42a3-4fb7-a3ca-2c6fef9c7f54 None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:31.140529 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 465/894] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jun 23 15:48:31 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 15:48:31.141297 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:31.141476 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51c562d-cc43-423e-813f-1f9ca0a165ff None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:31.141976 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51c562d-cc43-423e-813f-1f9ca0a165ff None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:31.141976 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51c562d-cc43-423e-813f-1f9ca0a165ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:31.159563 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-d51c562d-cc43-423e-813f-1f9ca0a165ff None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:48:31.161351 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-d51c562d-cc43-423e-813f-1f9ca0a165ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:48:31.193494 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81399027-fac6-4298-a4a5-da3ac801a918 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:31.230255 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087d55c7-16a9-4dfd-add3-251e80a80dab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:31.230955 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087d55c7-16a9-4dfd-add3-251e80a80dab None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:31.231244 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087d55c7-16a9-4dfd-add3-251e80a80dab None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:31.231524 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087d55c7-16a9-4dfd-add3-251e80a80dab None admin] PATH_INFO: `/v3/services` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:31.232240 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087d55c7-16a9-4dfd-add3-251e80a80dab None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:48:31.232555 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087d55c7-16a9-4dfd-add3-251e80a80dab None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:31.233922 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087d55c7-16a9-4dfd-add3-251e80a80dab None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:31.243731 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 430/895] 10.222.0.46 () {56 vars in 1126 bytes} [Thu Jun 23 15:48:31 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 15:48:31.243731 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc33b05-6d03-4327-881b-f343e29cdf4b None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:31.243731 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc33b05-6d03-4327-881b-f343e29cdf4b None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:31.243731 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc33b05-6d03-4327-881b-f343e29cdf4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:31.244107 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 466/896] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jun 23 15:48:31 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 15:48:31.245054 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51c562d-cc43-423e-813f-1f9ca0a165ff None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:31.261580 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-3cc33b05-6d03-4327-881b-f343e29cdf4b None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:48:31.262882 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-3cc33b05-6d03-4327-881b-f343e29cdf4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:48:31.338931 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 431/897] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jun 23 15:48:31 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 15:48:31.339630 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc33b05-6d03-4327-881b-f343e29cdf4b None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:31.355798 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:31.383949 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51c562d-cc43-423e-813f-1f9ca0a165ff None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:31.392939 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:31.423319 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1efad5-8d24-497e-9c63-2792ef36c6f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:31.423986 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1efad5-8d24-497e-9c63-2792ef36c6f2 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:31.424269 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1efad5-8d24-497e-9c63-2792ef36c6f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:31.424545 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1efad5-8d24-497e-9c63-2792ef36c6f2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:31.425763 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1efad5-8d24-497e-9c63-2792ef36c6f2 None admin] RBAC: Adding query filter params (interface=public, service_id=2c42432e733c4334a94b2bcf3661ad38, region_id=RegionOne) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:48:31.426124 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1efad5-8d24-497e-9c63-2792ef36c6f2 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:31.427879 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1efad5-8d24-497e-9c63-2792ef36c6f2 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:31.433987 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 467/898] 10.222.0.46 () {56 vars in 1251 bytes} [Thu Jun 23 15:48:31 2022] GET /identity/v3/endpoints?service_id=2c42432e733c4334a94b2bcf3661ad38&interface=public®ion_id=RegionOne => generated 194 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:48:31.435968 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1efad5-8d24-497e-9c63-2792ef36c6f2 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:31.450846 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc33b05-6d03-4327-881b-f343e29cdf4b None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:31.487855 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6da2c38-3c2c-4597-a39b-2b9f48009ef1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:31.488929 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6da2c38-3c2c-4597-a39b-2b9f48009ef1 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:31.489221 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6da2c38-3c2c-4597-a39b-2b9f48009ef1 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:31.489876 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6da2c38-3c2c-4597-a39b-2b9f48009ef1 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:31.503722 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6da2c38-3c2c-4597-a39b-2b9f48009ef1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 15:48:31.513093 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-f6da2c38-3c2c-4597-a39b-2b9f48009ef1 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 23 15:48:31.513093 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:48:31.513093 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:48:31.513093 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:48:31.513093 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:48:31.513093 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:48:31.513093 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:48:31.513093 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:48:31.513093 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:48:31.513093 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:48:31.513093 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:48:31.513093 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:48:31.513093 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 15:48:31.513093 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 15:48:31.513093 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 15:48:31.513093 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 15:48:31.514039 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 15:48:31.514039 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 15:48:31.514039 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 15:48:31.514039 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 15:48:31.514039 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:48:31.514039 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:48:31.514039 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:48:31.514039 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:48:31.514039 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:48:31.514039 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:48:31.514039 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:48:31.514039 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Jun 23 15:48:31.514039 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:48:31.514039 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:48:31.514039 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:48:31.514039 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:48:31.514915 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:48:31.514915 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:48:31.514915 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:48:31.514915 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:48:31.514915 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 15:48:31.514915 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 15:48:31.514915 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 15:48:31.514915 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 15:48:31.514915 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 15:48:31.514915 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 15:48:31.514915 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 23 15:48:31.514915 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Jun 23 15:48:31.514915 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 432/899] 10.222.0.46 () {56 vars in 1120 bytes} [Thu Jun 23 15:48:31 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:48:31.515648 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6da2c38-3c2c-4597-a39b-2b9f48009ef1 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:31.548168 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1efad5-8d24-497e-9c63-2792ef36c6f2 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:31.591003 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccbbafa-16b6-4291-8822-6a60fe2d16f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:31.591681 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccbbafa-16b6-4291-8822-6a60fe2d16f8 None admin] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:31.591970 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccbbafa-16b6-4291-8822-6a60fe2d16f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:31.592251 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccbbafa-16b6-4291-8822-6a60fe2d16f8 None admin] PATH_INFO: `/v3/services` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:31.592985 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccbbafa-16b6-4291-8822-6a60fe2d16f8 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:31.594355 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccbbafa-16b6-4291-8822-6a60fe2d16f8 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:31.607865 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 468/900] 10.222.0.46 () {60 vars in 1141 bytes} [Thu Jun 23 15:48:31 2022] POST /identity/v3/services => generated 241 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 15:48:31.608678 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccbbafa-16b6-4291-8822-6a60fe2d16f8 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:31.611419 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:31.643782 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6da2c38-3c2c-4597-a39b-2b9f48009ef1 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:31.661552 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:31.706348 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d52f82-2449-4ccf-90b6-b3cb7a56eed1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:31.708061 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d52f82-2449-4ccf-90b6-b3cb7a56eed1 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:31.708428 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d52f82-2449-4ccf-90b6-b3cb7a56eed1 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:31.708785 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d52f82-2449-4ccf-90b6-b3cb7a56eed1 None admin] PATH_INFO: `/v3/services` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:31.709613 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d52f82-2449-4ccf-90b6-b3cb7a56eed1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:31.712777 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d52f82-2449-4ccf-90b6-b3cb7a56eed1 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:31.730429 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 433/901] 10.222.0.46 () {56 vars in 1087 bytes} [Thu Jun 23 15:48:31 2022] GET /identity/v3/services => generated 1497 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 15:48:31.743086 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccbbafa-16b6-4291-8822-6a60fe2d16f8 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:31.794309 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5a9840-c551-4348-8a58-a3540435d6ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:31.795093 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5a9840-c551-4348-8a58-a3540435d6ef None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:31.795387 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5a9840-c551-4348-8a58-a3540435d6ef None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:31.795668 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5a9840-c551-4348-8a58-a3540435d6ef None admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:31.796492 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5a9840-c551-4348-8a58-a3540435d6ef None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:48:31.796806 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5a9840-c551-4348-8a58-a3540435d6ef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:31.798246 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5a9840-c551-4348-8a58-a3540435d6ef None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:31.809093 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 469/902] 10.222.0.46 () {56 vars in 1115 bytes} [Thu Jun 23 15:48:31 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:48:31.814743 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d52f82-2449-4ccf-90b6-b3cb7a56eed1 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:31.929114 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d52f82-2449-4ccf-90b6-b3cb7a56eed1 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:31.965359 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c9123f-71b3-47c3-b08f-0d3aff6f010b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:31.966169 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c9123f-71b3-47c3-b08f-0d3aff6f010b None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:31.966453 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c9123f-71b3-47c3-b08f-0d3aff6f010b None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:31.966750 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c9123f-71b3-47c3-b08f-0d3aff6f010b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:31.978058 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c9123f-71b3-47c3-b08f-0d3aff6f010b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:31.979535 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c9123f-71b3-47c3-b08f-0d3aff6f010b None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:31.997837 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-a4c9123f-71b3-47c3-b08f-0d3aff6f010b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 15:48:31.997837 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:48:31.997837 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:48:31.997837 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:48:31.997837 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:48:31.997837 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:48:31.997837 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:48:31.997837 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:48:31.997837 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:48:31.997837 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:48:31.997837 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:48:31.997837 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:48:31.997837 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 15:48:31.997837 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 15:48:31.997837 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 15:48:31.998823 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 15:48:31.998823 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:48:31.998823 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:48:31.998823 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:48:31.998823 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:48:31.998823 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:48:31.998823 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:48:31.998823 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:48:31.998823 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Jun 23 15:48:31.998823 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:48:31.998823 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:48:31.998823 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:48:31.998823 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:48:31.998823 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:48:31.998823 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:48:31.998823 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:48:31.998823 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:48:31.999725 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 15:48:31.999725 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 15:48:31.999725 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 15:48:31.999725 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Jun 23 15:48:31.999725 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 434/903] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Jun 23 15:48:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:48:32.005610 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5a9840-c551-4348-8a58-a3540435d6ef None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:32.083541 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:32.124742 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5a9840-c551-4348-8a58-a3540435d6ef None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:32.163587 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b066e996-0ef4-4077-a237-a6a8ef1baa78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:32.164652 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b066e996-0ef4-4077-a237-a6a8ef1baa78 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:32.165150 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b066e996-0ef4-4077-a237-a6a8ef1baa78 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:32.165650 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b066e996-0ef4-4077-a237-a6a8ef1baa78 None admin] PATH_INFO: `/v3/domains` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:32.166591 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b066e996-0ef4-4077-a237-a6a8ef1baa78 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:48:32.167127 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b066e996-0ef4-4077-a237-a6a8ef1baa78 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:32.168726 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b066e996-0ef4-4077-a237-a6a8ef1baa78 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:32.182874 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 470/904] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Jun 23 15:48:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:48:32.190317 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c9123f-71b3-47c3-b08f-0d3aff6f010b None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:32.208130 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:32.254718 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178add86-2281-4427-b7eb-ec1f985a7b18 None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:32.255006 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178add86-2281-4427-b7eb-ec1f985a7b18 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:32.255291 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178add86-2281-4427-b7eb-ec1f985a7b18 None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:32.256679 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 471/905] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jun 23 15:48:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 15:48:32.262390 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d57f9da-18b9-4989-9cc9-67af3affb6c5 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:32.262680 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d57f9da-18b9-4989-9cc9-67af3affb6c5 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:32.262987 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d57f9da-18b9-4989-9cc9-67af3affb6c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:32.282455 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-3d57f9da-18b9-4989-9cc9-67af3affb6c5 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:48:32.283696 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-3d57f9da-18b9-4989-9cc9-67af3affb6c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:48:32.296025 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c9123f-71b3-47c3-b08f-0d3aff6f010b None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:32.340809 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895dab14-4827-4d50-a191-051e2086cca9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:32.342104 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895dab14-4827-4d50-a191-051e2086cca9 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:32.342630 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895dab14-4827-4d50-a191-051e2086cca9 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:32.343153 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895dab14-4827-4d50-a191-051e2086cca9 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:32.352894 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895dab14-4827-4d50-a191-051e2086cca9 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:32.354716 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895dab14-4827-4d50-a191-051e2086cca9 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:32.366851 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 472/906] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jun 23 15:48:32 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 23 15:48:32.372628 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-895dab14-4827-4d50-a191-051e2086cca9 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 23 15:48:32.372628 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:48:32.372628 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:48:32.372628 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:48:32.372628 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:48:32.372628 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:48:32.372628 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:48:32.372628 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:48:32.372628 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:48:32.372628 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:48:32.372628 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:48:32.372628 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:48:32.372628 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 15:48:32.372628 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 15:48:32.372628 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 15:48:32.372628 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 15:48:32.373570 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:48:32.373570 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:48:32.373570 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 15:48:32.373570 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 15:48:32.373570 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 15:48:32.373570 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 15:48:32.373570 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:48:32.373570 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:48:32.373570 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:48:32.373570 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:48:32.373570 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:48:32.373570 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Jun 23 15:48:32.373570 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:48:32.373570 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:48:32.373570 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:48:32.373570 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:48:32.373570 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:48:32.373570 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:48:32.375167 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:48:32.375167 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:48:32.375167 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 15:48:32.375167 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 15:48:32.375167 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 15:48:32.375167 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 15:48:32.375167 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 15:48:32.375167 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 15:48:32.375167 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 23 15:48:32.375167 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Jun 23 15:48:32.375167 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 435/907] 10.222.0.46 () {56 vars in 1117 bytes} [Thu Jun 23 15:48:32 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:48:32.380860 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d57f9da-18b9-4989-9cc9-67af3affb6c5 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:32.390471 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895dab14-4827-4d50-a191-051e2086cca9 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:32.445070 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:32.490382 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:32.498390 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d57f9da-18b9-4989-9cc9-67af3affb6c5 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:32.523203 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895dab14-4827-4d50-a191-051e2086cca9 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:32.536483 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5b8f28-51c1-4473-bd59-5f48cf97284f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:32.537245 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5b8f28-51c1-4473-bd59-5f48cf97284f None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:32.537431 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5b8f28-51c1-4473-bd59-5f48cf97284f None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:32.537691 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5b8f28-51c1-4473-bd59-5f48cf97284f None admin] PATH_INFO: `/v3/users` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:32.538533 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5b8f28-51c1-4473-bd59-5f48cf97284f None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:48:32.538863 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5b8f28-51c1-4473-bd59-5f48cf97284f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:32.540306 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5b8f28-51c1-4473-bd59-5f48cf97284f None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:32.561912 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 473/908] 10.222.0.46 () {56 vars in 1149 bytes} [Thu Jun 23 15:48:32 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:48:32.568412 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5b8f28-51c1-4473-bd59-5f48cf97284f None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:32.569156 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ccdb4c-e221-461f-9247-794260cee9b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:32.570142 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ccdb4c-e221-461f-9247-794260cee9b5 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:32.570584 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ccdb4c-e221-461f-9247-794260cee9b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:32.571480 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ccdb4c-e221-461f-9247-794260cee9b5 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:32.572358 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ccdb4c-e221-461f-9247-794260cee9b5 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:32.573936 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ccdb4c-e221-461f-9247-794260cee9b5 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:32.591325 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-04ccdb4c-e221-461f-9247-794260cee9b5 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 23 15:48:32.591325 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:48:32.591325 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 15:48:32.591325 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 15:48:32.591325 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 15:48:32.591325 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 15:48:32.591325 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 15:48:32.591325 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 15:48:32.591325 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 23 15:48:32.591325 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Jun 23 15:48:32.591325 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 15:48:32.591325 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Jun 23 15:48:32.591325 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:48:32.591325 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:48:32.591325 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:48:32.591325 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:48:32.591325 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:48:32.592387 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:48:32.592387 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:48:32.592387 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:48:32.592387 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:48:32.592387 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:48:32.592387 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:48:32.592387 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 15:48:32.592387 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 15:48:32.592387 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 15:48:32.592387 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 15:48:32.592387 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:48:32.592387 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:48:32.592387 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:48:32.592387 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:48:32.592387 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:48:32.592387 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:48:32.598841 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:48:32.598841 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Jun 23 15:48:32.598841 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:48:32.598841 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:48:32.598841 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:48:32.598841 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:48:32.598841 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:48:32.598841 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:48:32.598841 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:48:32.598841 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:48:32.598841 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 15:48:32.598841 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 15:48:32.598841 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 23 15:48:32.598841 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Jun 23 15:48:32.598841 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 436/909] 10.222.0.46 () {56 vars in 1114 bytes} [Thu Jun 23 15:48:32 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:48:32.601225 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ccdb4c-e221-461f-9247-794260cee9b5 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:32.686852 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5b8f28-51c1-4473-bd59-5f48cf97284f None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:32.701217 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:32.728006 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ccdb4c-e221-461f-9247-794260cee9b5 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:32.732384 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fe0194-a980-41a4-9ebc-45fff48fb6f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:32.733173 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fe0194-a980-41a4-9ebc-45fff48fb6f6 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:32.733562 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fe0194-a980-41a4-9ebc-45fff48fb6f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:32.733940 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fe0194-a980-41a4-9ebc-45fff48fb6f6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:32.769073 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7bf452-e7db-49db-9c7e-6f4098a896d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:32.770855 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7bf452-e7db-49db-9c7e-6f4098a896d3 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:32.771482 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7bf452-e7db-49db-9c7e-6f4098a896d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:32.772230 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7bf452-e7db-49db-9c7e-6f4098a896d3 None admin] PATH_INFO: `/v3/services` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:32.773288 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7bf452-e7db-49db-9c7e-6f4098a896d3 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:48:32.773753 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7bf452-e7db-49db-9c7e-6f4098a896d3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:32.775620 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7bf452-e7db-49db-9c7e-6f4098a896d3 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:32.786573 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 437/910] 10.222.0.46 () {56 vars in 1114 bytes} [Thu Jun 23 15:48:32 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 15:48:32.788076 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7bf452-e7db-49db-9c7e-6f4098a896d3 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:32.798340 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:32.836869 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fe0194-a980-41a4-9ebc-45fff48fb6f6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:32.838792 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fe0194-a980-41a4-9ebc-45fff48fb6f6 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:32.900827 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7bf452-e7db-49db-9c7e-6f4098a896d3 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:32.942302 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ea0695-fd4c-45f3-a879-037d374ae431 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:32.943057 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ea0695-fd4c-45f3-a879-037d374ae431 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:32.943418 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ea0695-fd4c-45f3-a879-037d374ae431 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:32.943631 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ea0695-fd4c-45f3-a879-037d374ae431 None admin] PATH_INFO: `/v3/services` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:32.944366 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ea0695-fd4c-45f3-a879-037d374ae431 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:48:32.944679 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ea0695-fd4c-45f3-a879-037d374ae431 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:32.946715 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ea0695-fd4c-45f3-a879-037d374ae431 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:32.954610 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 438/911] 10.222.0.46 () {56 vars in 1114 bytes} [Thu Jun 23 15:48:32 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 15:48:32.963994 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ea0695-fd4c-45f3-a879-037d374ae431 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:32.973279 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 474/912] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Jun 23 15:48:32 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 23 15:48:32.979520 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fe0194-a980-41a4-9ebc-45fff48fb6f6 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:32.995129 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:33.045340 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:33.079320 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ea0695-fd4c-45f3-a879-037d374ae431 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:33.092001 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fe0194-a980-41a4-9ebc-45fff48fb6f6 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:33.116011 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711478b5-e8c3-4471-9896-db6c4f04f295 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:33.116823 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711478b5-e8c3-4471-9896-db6c4f04f295 None admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:33.117214 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711478b5-e8c3-4471-9896-db6c4f04f295 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:33.117599 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711478b5-e8c3-4471-9896-db6c4f04f295 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:33.118542 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711478b5-e8c3-4471-9896-db6c4f04f295 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:33.120135 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711478b5-e8c3-4471-9896-db6c4f04f295 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:33.131129 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bcf2b5-f150-4b20-9b3b-1877268b86ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:33.132221 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bcf2b5-f150-4b20-9b3b-1877268b86ec None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:33.132630 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bcf2b5-f150-4b20-9b3b-1877268b86ec None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:33.132932 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bcf2b5-f150-4b20-9b3b-1877268b86ec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:33.141818 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 439/913] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Jun 23 15:48:32 2022] POST /identity/v3/endpoints => generated 346 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 15:48:33.146883 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85bcf2b5-f150-4b20-9b3b-1877268b86ec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:33.146883 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85bcf2b5-f150-4b20-9b3b-1877268b86ec None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:33.168358 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-85bcf2b5-f150-4b20-9b3b-1877268b86ec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 15:48:33.168358 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:48:33.168358 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:48:33.168358 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:48:33.168358 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:48:33.168358 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:48:33.168358 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:48:33.168358 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:48:33.168358 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:48:33.168358 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:48:33.168358 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:48:33.168358 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:48:33.168358 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 15:48:33.168358 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 15:48:33.168358 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 15:48:33.169509 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 15:48:33.169509 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:48:33.169509 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:48:33.169509 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:48:33.169509 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:48:33.169509 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:48:33.169509 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:48:33.169509 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:48:33.169509 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Jun 23 15:48:33.169509 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:48:33.169509 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:48:33.169509 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:48:33.169509 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:48:33.169509 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:48:33.169509 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:48:33.169509 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:48:33.169509 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:48:33.170719 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 15:48:33.170719 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 15:48:33.170719 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 15:48:33.170719 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Jun 23 15:48:33.172895 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 475/914] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Jun 23 15:48:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 15:48:33.180006 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711478b5-e8c3-4471-9896-db6c4f04f295 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:33.310445 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711478b5-e8c3-4471-9896-db6c4f04f295 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:33.336154 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:33.350284 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631c60a3-f663-4c30-a390-51324160d55a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:33.351594 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631c60a3-f663-4c30-a390-51324160d55a None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:33.351594 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631c60a3-f663-4c30-a390-51324160d55a None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:33.351594 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631c60a3-f663-4c30-a390-51324160d55a None admin] PATH_INFO: `/v3/domains` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:33.352250 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631c60a3-f663-4c30-a390-51324160d55a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:48:33.352556 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631c60a3-f663-4c30-a390-51324160d55a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:33.353929 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631c60a3-f663-4c30-a390-51324160d55a None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:33.369351 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 440/915] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Jun 23 15:48:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:48:33.375333 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bcf2b5-f150-4b20-9b3b-1877268b86ec None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:33.443548 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:33.484581 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bcf2b5-f150-4b20-9b3b-1877268b86ec None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:33.521240 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9811c2bc-243f-4ff1-a3cd-62659eb053f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:33.522067 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9811c2bc-243f-4ff1-a3cd-62659eb053f1 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:33.522352 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9811c2bc-243f-4ff1-a3cd-62659eb053f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:33.522622 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9811c2bc-243f-4ff1-a3cd-62659eb053f1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:33.530261 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9811c2bc-243f-4ff1-a3cd-62659eb053f1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:33.531739 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9811c2bc-243f-4ff1-a3cd-62659eb053f1 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:33.546469 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-9811c2bc-243f-4ff1-a3cd-62659eb053f1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 15:48:33.546469 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:48:33.546469 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:48:33.546469 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:48:33.546469 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:48:33.546469 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:48:33.546469 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:48:33.546469 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:48:33.546469 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:48:33.546469 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:48:33.546469 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:48:33.546469 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:48:33.546469 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 15:48:33.546469 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 15:48:33.546469 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 15:48:33.547450 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 15:48:33.547450 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:48:33.547450 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:48:33.547450 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:48:33.547450 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:48:33.547450 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:48:33.547450 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:48:33.547450 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:48:33.547450 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Jun 23 15:48:33.547450 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:48:33.547450 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:48:33.547450 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:48:33.547450 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:48:33.547450 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:48:33.547450 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:48:33.547450 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:48:33.547450 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:48:33.548317 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 15:48:33.548317 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 15:48:33.548317 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 15:48:33.548317 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 15:48:33.548317 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 15:48:33.548317 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 15:48:33.548317 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 15:48:33.548317 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Jun 23 15:48:33.548317 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 476/916] 10.222.0.46 () {56 vars in 1111 bytes} [Thu Jun 23 15:48:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:48:33.555165 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631c60a3-f663-4c30-a390-51324160d55a None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:33.673619 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631c60a3-f663-4c30-a390-51324160d55a None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:33.711266 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51af96da-0794-4f30-a187-6ffa419999b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:33.712331 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51af96da-0794-4f30-a187-6ffa419999b1 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:33.712829 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51af96da-0794-4f30-a187-6ffa419999b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:33.713307 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51af96da-0794-4f30-a187-6ffa419999b1 None admin] PATH_INFO: `/v3/projects` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:33.714313 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51af96da-0794-4f30-a187-6ffa419999b1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:48:33.714900 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51af96da-0794-4f30-a187-6ffa419999b1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:33.716709 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51af96da-0794-4f30-a187-6ffa419999b1 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:33.729958 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 441/917] 10.222.0.46 () {56 vars in 1148 bytes} [Thu Jun 23 15:48:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:48:33.736652 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9811c2bc-243f-4ff1-a3cd-62659eb053f1 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:33.766468 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b329c1-0117-4dea-9f45-7fad2a8b40ab None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:33.766825 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b329c1-0117-4dea-9f45-7fad2a8b40ab None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:33.767070 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b329c1-0117-4dea-9f45-7fad2a8b40ab None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:33.768575 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 442/918] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jun 23 15:48:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 15:48:33.774633 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d638152d-2c30-467b-9413-a5223dfd3288 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:33.775315 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d638152d-2c30-467b-9413-a5223dfd3288 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:33.775798 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d638152d-2c30-467b-9413-a5223dfd3288 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:33.793278 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-d638152d-2c30-467b-9413-a5223dfd3288 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:48:33.794881 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-d638152d-2c30-467b-9413-a5223dfd3288 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:48:33.833833 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:33.878743 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9811c2bc-243f-4ff1-a3cd-62659eb053f1 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:33.879471 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 443/919] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jun 23 15:48:33 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 23 15:48:33.893931 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d638152d-2c30-467b-9413-a5223dfd3288 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:33.896753 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:33.915013 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2dfba1-4368-4573-ae04-14f7dd4a11a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:33.915683 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2dfba1-4368-4573-ae04-14f7dd4a11a9 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:33.915986 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2dfba1-4368-4573-ae04-14f7dd4a11a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:33.916263 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2dfba1-4368-4573-ae04-14f7dd4a11a9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:33.917102 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2dfba1-4368-4573-ae04-14f7dd4a11a9 None admin] RBAC: Adding query filter params (role.id=50fafe10bef649b4b87dfab0e16e7915, scope.project.id=28ecdf5b474c44098c722f677d2a0b77, user.id=be48addfaa154555b8d0cb7aac8b32ac) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:48:33.917423 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2dfba1-4368-4573-ae04-14f7dd4a11a9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:33.919195 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2dfba1-4368-4573-ae04-14f7dd4a11a9 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:33.930774 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 477/920] 10.222.0.46 () {56 vars in 1374 bytes} [Thu Jun 23 15:48:33 2022] GET /identity/v3/role_assignments?user.id=be48addfaa154555b8d0cb7aac8b32ac&scope.project.id=28ecdf5b474c44098c722f677d2a0b77&role.id=50fafe10bef649b4b87dfab0e16e7915 => generated 259 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:48:33.958390 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb57ab84-3673-4cb1-8600-6cac38920119 None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:33.959049 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb57ab84-3673-4cb1-8600-6cac38920119 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:33.959530 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb57ab84-3673-4cb1-8600-6cac38920119 None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:33.961218 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 478/921] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jun 23 15:48:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 15:48:33.968345 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9220ef40-e4cf-40cf-9478-f71789591322 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:33.968974 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9220ef40-e4cf-40cf-9478-f71789591322 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:33.969869 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9220ef40-e4cf-40cf-9478-f71789591322 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:33.996256 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-9220ef40-e4cf-40cf-9478-f71789591322 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:48:33.997835 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-9220ef40-e4cf-40cf-9478-f71789591322 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:48:34.030123 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d638152d-2c30-467b-9413-a5223dfd3288 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:34.077078 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52418b3a-77f5-4f11-81b4-749573693cac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:34.079379 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52418b3a-77f5-4f11-81b4-749573693cac None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:34.079668 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52418b3a-77f5-4f11-81b4-749573693cac None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:34.079947 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52418b3a-77f5-4f11-81b4-749573693cac None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:34.080628 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52418b3a-77f5-4f11-81b4-749573693cac None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:34.082044 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52418b3a-77f5-4f11-81b4-749573693cac None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:34.098774 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 479/922] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jun 23 15:48:33 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 15:48:34.099679 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-52418b3a-77f5-4f11-81b4-749573693cac None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 23 15:48:34.099679 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:48:34.099679 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 15:48:34.099679 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 15:48:34.099679 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 15:48:34.099679 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 15:48:34.099679 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 15:48:34.099679 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 15:48:34.099679 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 23 15:48:34.099679 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Jun 23 15:48:34.099679 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 15:48:34.099679 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Jun 23 15:48:34.099679 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:48:34.099679 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:48:34.099679 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:48:34.099679 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:48:34.099679 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:48:34.100679 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:48:34.100679 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:48:34.100679 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:48:34.100679 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:48:34.100679 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:48:34.100679 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:48:34.100679 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 15:48:34.100679 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 15:48:34.100679 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 15:48:34.100679 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 15:48:34.100679 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:48:34.100679 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:48:34.100679 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:48:34.100679 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:48:34.100679 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:48:34.100679 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:48:34.101511 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:48:34.101511 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Jun 23 15:48:34.101511 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:48:34.101511 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:48:34.101511 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:48:34.101511 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:48:34.101511 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:48:34.101511 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:48:34.101511 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:48:34.101511 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:48:34.101511 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 15:48:34.101511 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 15:48:34.101511 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 23 15:48:34.101511 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Jun 23 15:48:34.105954 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 444/923] 10.222.0.46 () {56 vars in 1126 bytes} [Thu Jun 23 15:48:33 2022] GET /identity/v3/services/object-store => generated 93 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:48:34.108725 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9220ef40-e4cf-40cf-9478-f71789591322 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:34.123406 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52418b3a-77f5-4f11-81b4-749573693cac None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:34.171863 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:34.246040 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9220ef40-e4cf-40cf-9478-f71789591322 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:34.263080 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52418b3a-77f5-4f11-81b4-749573693cac None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:34.268692 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:34.290949 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353b8928-785d-4161-9d97-8b6b774e5674 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:34.291940 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353b8928-785d-4161-9d97-8b6b774e5674 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:34.292232 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353b8928-785d-4161-9d97-8b6b774e5674 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:34.292518 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353b8928-785d-4161-9d97-8b6b774e5674 None admin] PATH_INFO: `/v3/services` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:34.293666 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353b8928-785d-4161-9d97-8b6b774e5674 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:48:34.294151 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353b8928-785d-4161-9d97-8b6b774e5674 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:34.295911 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353b8928-785d-4161-9d97-8b6b774e5674 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:34.307782 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 480/924] 10.222.0.46 () {56 vars in 1122 bytes} [Thu Jun 23 15:48:34 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 15:48:34.308848 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1013242e-026a-474f-811d-0233c0642a63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:34.309687 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1013242e-026a-474f-811d-0233c0642a63 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:34.309973 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1013242e-026a-474f-811d-0233c0642a63 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:34.310251 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1013242e-026a-474f-811d-0233c0642a63 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:34.310947 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1013242e-026a-474f-811d-0233c0642a63 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:34.312081 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353b8928-785d-4161-9d97-8b6b774e5674 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:34.312322 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1013242e-026a-474f-811d-0233c0642a63 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:34.328868 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-1013242e-026a-474f-811d-0233c0642a63 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 23 15:48:34.328868 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:48:34.328868 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 15:48:34.328868 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 15:48:34.328868 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 15:48:34.328868 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 15:48:34.328868 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 15:48:34.328868 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 15:48:34.328868 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 23 15:48:34.328868 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Jun 23 15:48:34.328868 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 15:48:34.328868 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Jun 23 15:48:34.328868 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:48:34.328868 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:48:34.328868 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:48:34.328868 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:48:34.329884 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:48:34.329884 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:48:34.329884 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:48:34.329884 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:48:34.329884 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:48:34.329884 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:48:34.329884 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:48:34.329884 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 15:48:34.329884 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 15:48:34.329884 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 15:48:34.329884 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 15:48:34.329884 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:48:34.329884 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:48:34.329884 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:48:34.329884 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:48:34.329884 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:48:34.330765 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:48:34.330765 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:48:34.330765 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Jun 23 15:48:34.330765 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:48:34.330765 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:48:34.330765 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:48:34.330765 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:48:34.330765 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:48:34.330765 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:48:34.330765 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:48:34.330765 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:48:34.330765 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 15:48:34.330765 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 15:48:34.330765 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 23 15:48:34.330765 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Jun 23 15:48:34.334771 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 445/925] 10.222.0.46 () {56 vars in 1132 bytes} [Thu Jun 23 15:48:34 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:48:34.337085 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1013242e-026a-474f-811d-0233c0642a63 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:34.430310 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353b8928-785d-4161-9d97-8b6b774e5674 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:34.470158 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ff62fe-cffc-4559-8a7a-4d53c56cebf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:34.470591 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ff62fe-cffc-4559-8a7a-4d53c56cebf3 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:34.470907 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ff62fe-cffc-4559-8a7a-4d53c56cebf3 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:34.471186 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ff62fe-cffc-4559-8a7a-4d53c56cebf3 None admin] PATH_INFO: `/v3/services` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:34.471905 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ff62fe-cffc-4559-8a7a-4d53c56cebf3 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:48:34.472229 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ff62fe-cffc-4559-8a7a-4d53c56cebf3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:34.474250 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ff62fe-cffc-4559-8a7a-4d53c56cebf3 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:34.475418 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:34.479079 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1013242e-026a-474f-811d-0233c0642a63 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:34.485567 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 481/926] 10.222.0.46 () {56 vars in 1122 bytes} [Thu Jun 23 15:48:34 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 15:48:34.486639 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ff62fe-cffc-4559-8a7a-4d53c56cebf3 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:34.516312 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de447170-1966-4255-9ff0-68ca07568266 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:34.517376 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de447170-1966-4255-9ff0-68ca07568266 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:34.517886 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de447170-1966-4255-9ff0-68ca07568266 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:34.518366 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de447170-1966-4255-9ff0-68ca07568266 None admin] PATH_INFO: `/v3/services` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:34.519316 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de447170-1966-4255-9ff0-68ca07568266 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:48:34.519881 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de447170-1966-4255-9ff0-68ca07568266 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:34.521452 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de447170-1966-4255-9ff0-68ca07568266 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:34.531785 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 446/927] 10.222.0.46 () {56 vars in 1126 bytes} [Thu Jun 23 15:48:34 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:48:34.532964 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de447170-1966-4255-9ff0-68ca07568266 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:34.537980 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:34.607958 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ff62fe-cffc-4559-8a7a-4d53c56cebf3 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:34.639676 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de447170-1966-4255-9ff0-68ca07568266 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:34.651614 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55370363-8574-4acc-92f6-e7998161c2d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:34.652650 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55370363-8574-4acc-92f6-e7998161c2d3 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:34.653008 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55370363-8574-4acc-92f6-e7998161c2d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:34.653360 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55370363-8574-4acc-92f6-e7998161c2d3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:34.654486 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55370363-8574-4acc-92f6-e7998161c2d3 None admin] RBAC: Adding query filter params (interface=public, service_id=fe53c3c741d14b4c973ff1a6e00fde97, region_id=RegionOne) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:48:34.654902 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55370363-8574-4acc-92f6-e7998161c2d3 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:34.656998 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55370363-8574-4acc-92f6-e7998161c2d3 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:34.669053 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 482/928] 10.222.0.46 () {56 vars in 1251 bytes} [Thu Jun 23 15:48:34 2022] GET /identity/v3/endpoints?service_id=fe53c3c741d14b4c973ff1a6e00fde97&interface=public®ion_id=RegionOne => generated 194 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:48:34.670384 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55370363-8574-4acc-92f6-e7998161c2d3 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:34.685273 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bc7264-63b1-4217-b081-a6ec771b48fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:34.686741 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bc7264-63b1-4217-b081-a6ec771b48fb None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:34.687373 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bc7264-63b1-4217-b081-a6ec771b48fb None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:34.687938 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bc7264-63b1-4217-b081-a6ec771b48fb None admin] PATH_INFO: `/v3/services` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:34.688846 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bc7264-63b1-4217-b081-a6ec771b48fb None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:48:34.689620 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bc7264-63b1-4217-b081-a6ec771b48fb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:34.692051 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bc7264-63b1-4217-b081-a6ec771b48fb None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:34.699091 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 447/929] 10.222.0.46 () {56 vars in 1126 bytes} [Thu Jun 23 15:48:34 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:48:34.704743 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bc7264-63b1-4217-b081-a6ec771b48fb None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:34.735599 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:34.791171 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55370363-8574-4acc-92f6-e7998161c2d3 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:34.803666 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:34.815767 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bc7264-63b1-4217-b081-a6ec771b48fb None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:34.832963 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e041f8-6576-4590-8857-7c906e5dbb7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:34.833983 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e041f8-6576-4590-8857-7c906e5dbb7b None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:34.834492 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e041f8-6576-4590-8857-7c906e5dbb7b None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:34.835007 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e041f8-6576-4590-8857-7c906e5dbb7b None admin] PATH_INFO: `/v3/services` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:34.835898 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e041f8-6576-4590-8857-7c906e5dbb7b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:34.837501 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e041f8-6576-4590-8857-7c906e5dbb7b None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:34.851095 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 483/930] 10.222.0.46 () {56 vars in 1087 bytes} [Thu Jun 23 15:48:34 2022] GET /identity/v3/services => generated 1497 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 15:48:34.853907 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2fea80-b1ee-4743-be6c-93e51b3218b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:34.854635 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2fea80-b1ee-4743-be6c-93e51b3218b1 None admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:34.854965 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2fea80-b1ee-4743-be6c-93e51b3218b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:34.855245 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2fea80-b1ee-4743-be6c-93e51b3218b1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:34.856006 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2fea80-b1ee-4743-be6c-93e51b3218b1 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:34.857429 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2fea80-b1ee-4743-be6c-93e51b3218b1 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:34.881553 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 448/931] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Jun 23 15:48:34 2022] POST /identity/v3/endpoints => generated 347 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 15:48:35.355077 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa93986a-b401-4e2d-9e7e-596cd578b5d3 None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:35.355592 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa93986a-b401-4e2d-9e7e-596cd578b5d3 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:35.355592 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa93986a-b401-4e2d-9e7e-596cd578b5d3 None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:35.357239 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 484/932] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jun 23 15:48:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 15:48:35.362630 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6988b3-aedf-43ac-88f6-99cc59f3e1c9 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:35.362955 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6988b3-aedf-43ac-88f6-99cc59f3e1c9 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:35.363237 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6988b3-aedf-43ac-88f6-99cc59f3e1c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:35.380613 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-7b6988b3-aedf-43ac-88f6-99cc59f3e1c9 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:48:35.381801 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-7b6988b3-aedf-43ac-88f6-99cc59f3e1c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:48:35.458791 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 449/933] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jun 23 15:48:35 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 15:48:35.482024 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa93986a-b401-4e2d-9e7e-596cd578b5d3 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:35.611240 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa93986a-b401-4e2d-9e7e-596cd578b5d3 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:35.656036 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7554388f-8c69-4465-b90c-e62869859a36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:35.657154 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7554388f-8c69-4465-b90c-e62869859a36 None admin] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:35.657681 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7554388f-8c69-4465-b90c-e62869859a36 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:35.658173 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7554388f-8c69-4465-b90c-e62869859a36 None admin] PATH_INFO: `/v3/projects` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:35.659149 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7554388f-8c69-4465-b90c-e62869859a36 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:35.660874 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7554388f-8c69-4465-b90c-e62869859a36 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:35.662481 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: WARNING keystone.server.flask.common [None req-7554388f-8c69-4465-b90c-e62869859a36 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jun 23 15:48:35.682605 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.sql.core [None req-7554388f-8c69-4465-b90c-e62869859a36 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 23 15:48:35.682605 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jun 23 15:48:35.682605 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [parameters: {'id': '48277ca8d9494247ad605a95587487be', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jun 23 15:48:35.682605 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80153) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 23 15:48:35.700905 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-7554388f-8c69-4465-b90c-e62869859a36 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 48277ca8d9494247ad605a95587487be.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 48277ca8d9494247ad605a95587487be. Jun 23 15:48:35.700905 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:48:35.700905 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 23 15:48:35.700905 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 23 15:48:35.700905 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 23 15:48:35.700905 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 23 15:48:35.700905 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 23 15:48:35.700905 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application result = self._query(query) Jun 23 15:48:35.700905 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 23 15:48:35.700905 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application conn.query(q) Jun 23 15:48:35.700905 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 23 15:48:35.700905 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 23 15:48:35.700905 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 23 15:48:35.701781 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application result.read() Jun 23 15:48:35.701781 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 23 15:48:35.701781 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 23 15:48:35.701781 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 23 15:48:35.701781 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 23 15:48:35.701781 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 23 15:48:35.701781 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 23 15:48:35.701781 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 23 15:48:35.701781 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 23 15:48:35.701781 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 23 15:48:35.701781 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Jun 23 15:48:35.701781 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 23 15:48:35.701781 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Jun 23 15:48:35.701781 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:48:35.701781 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 23 15:48:35.701781 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 23 15:48:35.701781 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jun 23 15:48:35.701781 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return project_ref.to_dict() Jun 23 15:48:35.702738 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 23 15:48:35.702738 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application next(self.gen) Jun 23 15:48:35.702738 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 23 15:48:35.702738 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application yield resource Jun 23 15:48:35.702738 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 23 15:48:35.702738 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application next(self.gen) Jun 23 15:48:35.702738 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 23 15:48:35.702738 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application self.session.rollback() Jun 23 15:48:35.702738 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 23 15:48:35.702738 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application self.force_reraise() Jun 23 15:48:35.702738 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 23 15:48:35.702738 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise self.value Jun 23 15:48:35.702738 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 23 15:48:35.702738 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 23 15:48:35.702738 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 23 15:48:35.702738 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application session.commit() Jun 23 15:48:35.702738 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 23 15:48:35.703658 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 23 15:48:35.703658 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 23 15:48:35.703658 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application self._prepare_impl() Jun 23 15:48:35.703658 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 23 15:48:35.703658 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application self.session.flush() Jun 23 15:48:35.703658 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 23 15:48:35.703658 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application self._flush(objects) Jun 23 15:48:35.703658 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 23 15:48:35.703658 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 23 15:48:35.703658 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 23 15:48:35.703658 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application compat.raise_( Jun 23 15:48:35.703658 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 23 15:48:35.703658 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception Jun 23 15:48:35.703658 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 23 15:48:35.703658 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application flush_context.execute() Jun 23 15:48:35.703658 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 23 15:48:35.703658 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rec.execute(self) Jun 23 15:48:35.704547 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 23 15:48:35.704547 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 23 15:48:35.704547 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 23 15:48:35.704547 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 23 15:48:35.704547 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jun 23 15:48:35.704547 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application c = connection._execute_20( Jun 23 15:48:35.704547 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 23 15:48:35.704547 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 23 15:48:35.704547 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 23 15:48:35.704547 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 23 15:48:35.704547 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 23 15:48:35.704547 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 23 15:48:35.704547 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 23 15:48:35.704547 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 23 15:48:35.704547 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 23 15:48:35.705419 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 23 15:48:35.705419 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 23 15:48:35.705419 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception Jun 23 15:48:35.705419 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 23 15:48:35.705419 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 23 15:48:35.705419 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 23 15:48:35.705419 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 23 15:48:35.705419 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 23 15:48:35.705419 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application result = self._query(query) Jun 23 15:48:35.705419 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 23 15:48:35.705419 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application conn.query(q) Jun 23 15:48:35.705419 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 23 15:48:35.705419 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 23 15:48:35.705419 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 23 15:48:35.705419 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application result.read() Jun 23 15:48:35.705419 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 23 15:48:35.705419 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 23 15:48:35.706300 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 23 15:48:35.706300 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 23 15:48:35.706300 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 23 15:48:35.706300 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 23 15:48:35.706300 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 23 15:48:35.706300 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 23 15:48:35.706300 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 23 15:48:35.706300 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jun 23 15:48:35.706300 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [parameters: {'id': '48277ca8d9494247ad605a95587487be', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jun 23 15:48:35.706300 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 23 15:48:35.706300 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Jun 23 15:48:35.706300 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 15:48:35.706300 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Jun 23 15:48:35.706300 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:48:35.706300 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jun 23 15:48:35.708786 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jun 23 15:48:35.708786 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 23 15:48:35.708786 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 23 15:48:35.708786 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jun 23 15:48:35.708786 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Jun 23 15:48:35.708786 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 15:48:35.708786 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Jun 23 15:48:35.708786 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:48:35.708786 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:48:35.708786 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:48:35.708786 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:48:35.708786 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:48:35.708786 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:48:35.708786 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:48:35.708786 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:48:35.708786 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:48:35.708786 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:48:35.711170 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:48:35.711170 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jun 23 15:48:35.711170 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jun 23 15:48:35.711170 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:48:35.711170 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:48:35.711170 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jun 23 15:48:35.711170 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.Conflict( Jun 23 15:48:35.711170 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 48277ca8d9494247ad605a95587487be. Jun 23 15:48:35.711170 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Jun 23 15:48:35.714222 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 485/934] 10.222.0.46 () {60 vars in 1140 bytes} [Thu Jun 23 15:48:35 2022] POST /identity/v3/projects => generated 262 bytes in 233 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:48:35.720285 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6988b3-aedf-43ac-88f6-99cc59f3e1c9 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:35.747735 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:35.836494 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6988b3-aedf-43ac-88f6-99cc59f3e1c9 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:35.873243 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84779457-ba93-4ecc-909a-c315a8a6d388 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:35.874428 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84779457-ba93-4ecc-909a-c315a8a6d388 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:35.875023 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84779457-ba93-4ecc-909a-c315a8a6d388 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:35.875571 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84779457-ba93-4ecc-909a-c315a8a6d388 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:35.883590 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84779457-ba93-4ecc-909a-c315a8a6d388 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:35.885299 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84779457-ba93-4ecc-909a-c315a8a6d388 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:35.900453 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-84779457-ba93-4ecc-909a-c315a8a6d388 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 15:48:35.900453 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:48:35.900453 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:48:35.900453 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:48:35.900453 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:48:35.900453 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:48:35.900453 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:48:35.900453 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:48:35.900453 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:48:35.900453 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:48:35.900453 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:48:35.900453 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:48:35.900453 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 15:48:35.900453 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 15:48:35.900453 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 15:48:35.901381 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 15:48:35.901381 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:48:35.901381 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:48:35.901381 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:48:35.901381 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:48:35.901381 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:48:35.901381 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:48:35.901381 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:48:35.901381 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Jun 23 15:48:35.901381 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:48:35.901381 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:48:35.901381 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:48:35.901381 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:48:35.901381 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:48:35.901381 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:48:35.901381 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:48:35.901381 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:48:35.902297 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 15:48:35.902297 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 15:48:35.902297 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 15:48:35.902297 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 15:48:35.902297 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 15:48:35.902297 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 15:48:35.902297 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 15:48:35.902297 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Jun 23 15:48:35.904316 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 450/935] 10.222.0.46 () {56 vars in 1111 bytes} [Thu Jun 23 15:48:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:48:35.912078 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7554388f-8c69-4465-b90c-e62869859a36 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:35.949939 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:36.030906 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7554388f-8c69-4465-b90c-e62869859a36 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:36.044509 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8136c5-bd2d-4908-a728-5106e96f87ff None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:36.044962 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8136c5-bd2d-4908-a728-5106e96f87ff None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:36.045343 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8136c5-bd2d-4908-a728-5106e96f87ff None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:36.047296 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 451/936] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jun 23 15:48:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 15:48:36.054473 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bccc22-9fe4-415d-9b97-bd898023d27b None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:36.054911 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bccc22-9fe4-415d-9b97-bd898023d27b None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:36.055291 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bccc22-9fe4-415d-9b97-bd898023d27b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:36.072673 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-09bccc22-9fe4-415d-9b97-bd898023d27b None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:48:36.073185 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0aeaa2-2158-4860-a40b-03555743a92a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:36.074022 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0aeaa2-2158-4860-a40b-03555743a92a None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:36.074227 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-09bccc22-9fe4-415d-9b97-bd898023d27b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:48:36.074604 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0aeaa2-2158-4860-a40b-03555743a92a None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:36.075070 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0aeaa2-2158-4860-a40b-03555743a92a None admin] PATH_INFO: `/v3/projects` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:36.076001 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0aeaa2-2158-4860-a40b-03555743a92a None admin] RBAC: Adding query filter params (name=service) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:48:36.076487 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0aeaa2-2158-4860-a40b-03555743a92a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:36.079543 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0aeaa2-2158-4860-a40b-03555743a92a None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:36.111472 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 486/937] 10.222.0.46 () {56 vars in 1112 bytes} [Thu Jun 23 15:48:35 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:48:36.152458 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:36.158390 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 452/938] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jun 23 15:48:36 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 15:48:36.183160 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0aeaa2-2158-4860-a40b-03555743a92a None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:36.284475 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:36.297019 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0aeaa2-2158-4860-a40b-03555743a92a None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:36.332836 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57834073-0bbd-4853-a304-80cded158e4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:36.333756 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57834073-0bbd-4853-a304-80cded158e4d None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:36.334116 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57834073-0bbd-4853-a304-80cded158e4d None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:36.334465 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57834073-0bbd-4853-a304-80cded158e4d None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:36.345427 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57834073-0bbd-4853-a304-80cded158e4d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 15:48:36.354240 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-57834073-0bbd-4853-a304-80cded158e4d None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 23 15:48:36.354240 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:48:36.354240 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:48:36.354240 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:48:36.354240 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:48:36.354240 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:48:36.354240 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:48:36.354240 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:48:36.354240 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:48:36.354240 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:48:36.354240 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:48:36.354240 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:48:36.354240 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 15:48:36.354240 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 15:48:36.354240 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 15:48:36.354240 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 15:48:36.355629 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 15:48:36.355629 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 15:48:36.355629 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 15:48:36.355629 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 15:48:36.355629 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:48:36.355629 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:48:36.355629 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:48:36.355629 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:48:36.355629 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:48:36.355629 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:48:36.355629 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:48:36.355629 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Jun 23 15:48:36.355629 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:48:36.355629 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:48:36.355629 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:48:36.355629 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:48:36.357032 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:48:36.357032 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:48:36.357032 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:48:36.357032 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:48:36.357032 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 15:48:36.357032 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 15:48:36.357032 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 15:48:36.357032 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 15:48:36.357032 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 15:48:36.357032 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 15:48:36.357032 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 23 15:48:36.357032 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Jun 23 15:48:36.357032 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 487/939] 10.222.0.46 () {56 vars in 1120 bytes} [Thu Jun 23 15:48:36 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:48:36.361788 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bccc22-9fe4-415d-9b97-bd898023d27b None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:36.467147 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bccc22-9fe4-415d-9b97-bd898023d27b None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:36.502939 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951e35b2-83b7-4a19-b18c-844b4fd71120 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:36.504131 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951e35b2-83b7-4a19-b18c-844b4fd71120 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:36.504642 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951e35b2-83b7-4a19-b18c-844b4fd71120 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:36.505111 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951e35b2-83b7-4a19-b18c-844b4fd71120 None admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:36.506049 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951e35b2-83b7-4a19-b18c-844b4fd71120 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:48:36.506548 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951e35b2-83b7-4a19-b18c-844b4fd71120 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:36.508453 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951e35b2-83b7-4a19-b18c-844b4fd71120 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:36.518869 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 453/940] 10.222.0.46 () {56 vars in 1115 bytes} [Thu Jun 23 15:48:36 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:48:36.528323 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57834073-0bbd-4853-a304-80cded158e4d None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:36.633890 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57834073-0bbd-4853-a304-80cded158e4d None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:36.670284 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be3fad2-f97e-457b-81c1-84826783a1d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:36.671496 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be3fad2-f97e-457b-81c1-84826783a1d2 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:36.671977 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be3fad2-f97e-457b-81c1-84826783a1d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:36.672443 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be3fad2-f97e-457b-81c1-84826783a1d2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:36.684194 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be3fad2-f97e-457b-81c1-84826783a1d2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:36.685947 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be3fad2-f97e-457b-81c1-84826783a1d2 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:36.704196 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-1be3fad2-f97e-457b-81c1-84826783a1d2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 15:48:36.704196 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:48:36.704196 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:48:36.704196 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:48:36.704196 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:48:36.704196 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:48:36.704196 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:48:36.704196 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:48:36.704196 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:48:36.704196 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:48:36.704196 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:48:36.704196 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:48:36.704196 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 15:48:36.704196 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 15:48:36.704196 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 15:48:36.705119 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 15:48:36.705119 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:48:36.705119 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:48:36.705119 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:48:36.705119 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:48:36.705119 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:48:36.705119 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:48:36.705119 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:48:36.705119 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Jun 23 15:48:36.705119 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:48:36.705119 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:48:36.705119 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:48:36.705119 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:48:36.705119 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:48:36.705119 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:48:36.705119 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:48:36.705119 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:48:36.706006 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 15:48:36.706006 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 15:48:36.706006 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 15:48:36.706006 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Jun 23 15:48:36.708884 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 488/941] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Jun 23 15:48:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:48:36.718829 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951e35b2-83b7-4a19-b18c-844b4fd71120 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:36.787528 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:36.835573 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951e35b2-83b7-4a19-b18c-844b4fd71120 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:36.871972 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1a279a-4538-47b0-b713-b099fd030a51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:36.873104 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1a279a-4538-47b0-b713-b099fd030a51 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:36.873588 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1a279a-4538-47b0-b713-b099fd030a51 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:36.874103 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1a279a-4538-47b0-b713-b099fd030a51 None admin] PATH_INFO: `/v3/domains` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:36.875058 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1a279a-4538-47b0-b713-b099fd030a51 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:48:36.875568 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1a279a-4538-47b0-b713-b099fd030a51 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:36.877210 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1a279a-4538-47b0-b713-b099fd030a51 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:36.891574 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 454/942] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Jun 23 15:48:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:48:36.897941 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be3fad2-f97e-457b-81c1-84826783a1d2 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:36.901454 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:36.973417 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048ab55d-07c6-4a2b-ae97-8b73bb56ed15 None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:36.973642 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048ab55d-07c6-4a2b-ae97-8b73bb56ed15 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:36.973922 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048ab55d-07c6-4a2b-ae97-8b73bb56ed15 None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:36.975666 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 455/943] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jun 23 15:48:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 15:48:36.981254 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69200769-e9db-470a-9fa7-e989662fa0bb None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:36.981534 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69200769-e9db-470a-9fa7-e989662fa0bb None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:36.981811 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69200769-e9db-470a-9fa7-e989662fa0bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:36.999620 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-69200769-e9db-470a-9fa7-e989662fa0bb None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:48:37.001017 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-69200769-e9db-470a-9fa7-e989662fa0bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:48:37.013415 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be3fad2-f97e-457b-81c1-84826783a1d2 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:37.050288 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae561e22-2c72-4eaa-807d-ae9217a4c0ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:37.051632 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae561e22-2c72-4eaa-807d-ae9217a4c0ac None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:37.052124 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae561e22-2c72-4eaa-807d-ae9217a4c0ac None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:37.052546 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae561e22-2c72-4eaa-807d-ae9217a4c0ac None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:37.063809 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae561e22-2c72-4eaa-807d-ae9217a4c0ac None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:37.065409 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae561e22-2c72-4eaa-807d-ae9217a4c0ac None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:37.081741 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 456/944] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jun 23 15:48:36 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 15:48:37.084720 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-ae561e22-2c72-4eaa-807d-ae9217a4c0ac None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 23 15:48:37.084720 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:48:37.084720 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:48:37.084720 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:48:37.084720 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:48:37.084720 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:48:37.084720 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:48:37.084720 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:48:37.084720 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:48:37.084720 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:48:37.084720 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:48:37.084720 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:48:37.084720 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 15:48:37.084720 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 15:48:37.084720 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 15:48:37.084720 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 15:48:37.085681 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:48:37.085681 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:48:37.085681 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 15:48:37.085681 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 15:48:37.085681 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 15:48:37.085681 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 15:48:37.085681 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:48:37.085681 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:48:37.085681 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:48:37.085681 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:48:37.085681 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:48:37.085681 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Jun 23 15:48:37.085681 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:48:37.085681 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:48:37.085681 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:48:37.085681 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:48:37.085681 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:48:37.085681 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:48:37.086611 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:48:37.086611 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:48:37.086611 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 15:48:37.086611 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 15:48:37.086611 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 15:48:37.086611 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 15:48:37.086611 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 15:48:37.086611 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 15:48:37.086611 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 23 15:48:37.086611 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Jun 23 15:48:37.088769 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 489/945] 10.222.0.46 () {56 vars in 1117 bytes} [Thu Jun 23 15:48:36 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:48:37.098725 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69200769-e9db-470a-9fa7-e989662fa0bb None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:37.100743 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d23a8d-c3a6-4bb7-94d3-7b32baf95541 None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:37.100743 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d23a8d-c3a6-4bb7-94d3-7b32baf95541 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:37.100743 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d23a8d-c3a6-4bb7-94d3-7b32baf95541 None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:37.105978 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 490/946] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jun 23 15:48:37 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 15:48:37.107247 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44035bfa-9e17-4627-a03b-e8fb9ee185ca None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:37.107540 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44035bfa-9e17-4627-a03b-e8fb9ee185ca None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:37.107823 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44035bfa-9e17-4627-a03b-e8fb9ee185ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:37.128444 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-44035bfa-9e17-4627-a03b-e8fb9ee185ca None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:48:37.130704 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-44035bfa-9e17-4627-a03b-e8fb9ee185ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:48:37.152533 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:37.190220 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:37.202296 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 491/947] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jun 23 15:48:37 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 15:48:37.207417 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44035bfa-9e17-4627-a03b-e8fb9ee185ca None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:37.209849 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69200769-e9db-470a-9fa7-e989662fa0bb None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:37.262097 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17d4609-50b0-42c5-8498-6e07d67c04a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:37.262758 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17d4609-50b0-42c5-8498-6e07d67c04a9 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:37.263041 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17d4609-50b0-42c5-8498-6e07d67c04a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:37.263315 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17d4609-50b0-42c5-8498-6e07d67c04a9 None admin] PATH_INFO: `/v3/users` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:37.264155 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17d4609-50b0-42c5-8498-6e07d67c04a9 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:48:37.264463 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17d4609-50b0-42c5-8498-6e07d67c04a9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:37.265892 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17d4609-50b0-42c5-8498-6e07d67c04a9 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:37.289480 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 457/948] 10.222.0.46 () {56 vars in 1149 bytes} [Thu Jun 23 15:48:37 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:48:37.289480 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17d4609-50b0-42c5-8498-6e07d67c04a9 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:37.317712 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44035bfa-9e17-4627-a03b-e8fb9ee185ca None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:37.359284 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0616620-2a27-4177-8aa2-03df4b11c55f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:37.360201 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0616620-2a27-4177-8aa2-03df4b11c55f None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:37.360496 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0616620-2a27-4177-8aa2-03df4b11c55f None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:37.360768 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0616620-2a27-4177-8aa2-03df4b11c55f None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:37.361428 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0616620-2a27-4177-8aa2-03df4b11c55f None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:37.362826 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0616620-2a27-4177-8aa2-03df4b11c55f None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:37.375060 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-d0616620-2a27-4177-8aa2-03df4b11c55f None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 23 15:48:37.375060 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:48:37.375060 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 15:48:37.375060 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 15:48:37.375060 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 15:48:37.375060 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 15:48:37.375060 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 15:48:37.375060 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 15:48:37.375060 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 23 15:48:37.375060 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Jun 23 15:48:37.375060 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 15:48:37.375060 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Jun 23 15:48:37.375060 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:48:37.375060 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:48:37.375060 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:48:37.375060 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:48:37.375060 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:48:37.376068 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:48:37.376068 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:48:37.376068 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:48:37.376068 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:48:37.376068 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:48:37.376068 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:48:37.376068 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 15:48:37.376068 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 15:48:37.376068 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 15:48:37.376068 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 15:48:37.376068 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:48:37.376068 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:48:37.376068 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:48:37.376068 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:48:37.376068 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:48:37.376068 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:48:37.376877 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:48:37.376877 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Jun 23 15:48:37.376877 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:48:37.376877 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:48:37.376877 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:48:37.376877 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:48:37.376877 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:48:37.376877 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:48:37.376877 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:48:37.376877 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:48:37.376877 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 15:48:37.376877 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 15:48:37.376877 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 23 15:48:37.376877 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Jun 23 15:48:37.376877 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 492/949] 10.222.0.46 () {56 vars in 1126 bytes} [Thu Jun 23 15:48:37 2022] GET /identity/v3/services/object-store => generated 93 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:48:37.378179 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0616620-2a27-4177-8aa2-03df4b11c55f None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:37.398791 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17d4609-50b0-42c5-8498-6e07d67c04a9 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:37.410992 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:37.436405 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f01522-7f94-4ae6-9fe1-2ac5f66ecf85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:37.437217 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f01522-7f94-4ae6-9fe1-2ac5f66ecf85 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:37.437507 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f01522-7f94-4ae6-9fe1-2ac5f66ecf85 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:37.437791 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f01522-7f94-4ae6-9fe1-2ac5f66ecf85 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:37.438456 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f01522-7f94-4ae6-9fe1-2ac5f66ecf85 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:37.439879 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f01522-7f94-4ae6-9fe1-2ac5f66ecf85 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:37.445440 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:37.452148 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-34f01522-7f94-4ae6-9fe1-2ac5f66ecf85 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 23 15:48:37.452148 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:48:37.452148 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 15:48:37.452148 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 15:48:37.452148 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 15:48:37.452148 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 15:48:37.452148 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 15:48:37.452148 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 15:48:37.452148 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 23 15:48:37.452148 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Jun 23 15:48:37.452148 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 15:48:37.452148 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Jun 23 15:48:37.452148 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:48:37.452148 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:48:37.452148 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:48:37.452148 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:48:37.452148 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:48:37.453116 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:48:37.453116 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:48:37.453116 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:48:37.453116 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:48:37.453116 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:48:37.453116 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:48:37.453116 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 15:48:37.453116 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 15:48:37.453116 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 15:48:37.453116 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 15:48:37.453116 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:48:37.453116 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:48:37.453116 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:48:37.453116 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:48:37.453116 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:48:37.453116 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:48:37.453952 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:48:37.453952 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Jun 23 15:48:37.453952 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:48:37.453952 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:48:37.453952 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:48:37.453952 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:48:37.453952 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:48:37.453952 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:48:37.453952 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:48:37.453952 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:48:37.453952 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 15:48:37.453952 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 15:48:37.453952 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 23 15:48:37.453952 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Jun 23 15:48:37.453952 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 458/950] 10.222.0.46 () {56 vars in 1111 bytes} [Thu Jun 23 15:48:37 2022] GET /identity/v3/services/compute => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:48:37.455391 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f01522-7f94-4ae6-9fe1-2ac5f66ecf85 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:37.498215 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0616620-2a27-4177-8aa2-03df4b11c55f None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:37.534415 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b266da-82dd-4b6c-b8c1-b7e09e685432 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:37.535353 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b266da-82dd-4b6c-b8c1-b7e09e685432 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:37.535739 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b266da-82dd-4b6c-b8c1-b7e09e685432 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:37.536120 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b266da-82dd-4b6c-b8c1-b7e09e685432 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:37.547365 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b266da-82dd-4b6c-b8c1-b7e09e685432 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:37.548933 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b266da-82dd-4b6c-b8c1-b7e09e685432 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:37.570222 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-d0b266da-82dd-4b6c-b8c1-b7e09e685432 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 15:48:37.570222 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:48:37.570222 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:48:37.570222 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:48:37.570222 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:48:37.570222 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:48:37.570222 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:48:37.570222 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:48:37.570222 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:48:37.570222 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:48:37.570222 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:48:37.570222 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:48:37.570222 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 15:48:37.570222 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 15:48:37.570222 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 15:48:37.571478 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 15:48:37.571478 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:48:37.571478 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:48:37.571478 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:48:37.571478 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:48:37.571478 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:48:37.571478 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:48:37.571478 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:48:37.571478 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Jun 23 15:48:37.571478 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:48:37.571478 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:48:37.571478 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:48:37.571478 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:48:37.571478 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:48:37.571478 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:48:37.571478 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:48:37.571478 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:48:37.572849 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 15:48:37.572849 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 15:48:37.572849 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 15:48:37.572849 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Jun 23 15:48:37.578753 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 493/951] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Jun 23 15:48:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:48:37.578753 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b266da-82dd-4b6c-b8c1-b7e09e685432 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:37.579514 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f01522-7f94-4ae6-9fe1-2ac5f66ecf85 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:37.615747 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4304059-603d-4fbd-a411-2b7b8bd77ec7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:37.616143 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4304059-603d-4fbd-a411-2b7b8bd77ec7 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:37.616433 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4304059-603d-4fbd-a411-2b7b8bd77ec7 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:37.616712 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4304059-603d-4fbd-a411-2b7b8bd77ec7 None admin] PATH_INFO: `/v3/services` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:37.617424 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4304059-603d-4fbd-a411-2b7b8bd77ec7 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:48:37.617735 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4304059-603d-4fbd-a411-2b7b8bd77ec7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:37.619140 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4304059-603d-4fbd-a411-2b7b8bd77ec7 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:37.629335 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 459/952] 10.222.0.46 () {56 vars in 1122 bytes} [Thu Jun 23 15:48:37 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:48:37.629335 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4304059-603d-4fbd-a411-2b7b8bd77ec7 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:37.672242 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:37.695618 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b266da-82dd-4b6c-b8c1-b7e09e685432 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:37.705333 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:37.733918 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8f4ae3-4dd2-461a-89ee-2499e0843d23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:37.734682 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8f4ae3-4dd2-461a-89ee-2499e0843d23 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:37.734853 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4304059-603d-4fbd-a411-2b7b8bd77ec7 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:37.734996 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8f4ae3-4dd2-461a-89ee-2499e0843d23 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:37.735399 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8f4ae3-4dd2-461a-89ee-2499e0843d23 None admin] PATH_INFO: `/v3/services` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:37.736177 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8f4ae3-4dd2-461a-89ee-2499e0843d23 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:48:37.736491 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8f4ae3-4dd2-461a-89ee-2499e0843d23 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:37.737876 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8f4ae3-4dd2-461a-89ee-2499e0843d23 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:37.748035 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 494/953] 10.222.0.46 () {56 vars in 1112 bytes} [Thu Jun 23 15:48:37 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:48:37.748035 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8f4ae3-4dd2-461a-89ee-2499e0843d23 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:37.775549 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2350f6-c66d-4240-b627-834cf83160f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:37.777615 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2350f6-c66d-4240-b627-834cf83160f1 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:37.778088 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2350f6-c66d-4240-b627-834cf83160f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:37.778366 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2350f6-c66d-4240-b627-834cf83160f1 None admin] PATH_INFO: `/v3/domains` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:37.779118 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2350f6-c66d-4240-b627-834cf83160f1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:48:37.779426 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2350f6-c66d-4240-b627-834cf83160f1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:37.780794 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2350f6-c66d-4240-b627-834cf83160f1 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:37.793903 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 460/954] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Jun 23 15:48:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:48:37.794652 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2350f6-c66d-4240-b627-834cf83160f1 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:37.864294 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8f4ae3-4dd2-461a-89ee-2499e0843d23 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:37.897997 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:37.901863 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2350f6-c66d-4240-b627-834cf83160f1 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:37.902085 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b7b099-fee7-4572-9f51-a02d79116366 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:37.902989 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b7b099-fee7-4572-9f51-a02d79116366 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:37.904094 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b7b099-fee7-4572-9f51-a02d79116366 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:37.904646 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b7b099-fee7-4572-9f51-a02d79116366 None admin] PATH_INFO: `/v3/services` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:37.905509 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b7b099-fee7-4572-9f51-a02d79116366 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:48:37.905966 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b7b099-fee7-4572-9f51-a02d79116366 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:37.907531 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b7b099-fee7-4572-9f51-a02d79116366 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:37.912998 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 495/955] 10.222.0.46 () {56 vars in 1122 bytes} [Thu Jun 23 15:48:37 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:48:37.914104 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b7b099-fee7-4572-9f51-a02d79116366 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:37.944973 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d11e630-cedf-46aa-a1d1-253c9bd7dd8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:37.946066 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d11e630-cedf-46aa-a1d1-253c9bd7dd8f None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:37.946368 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d11e630-cedf-46aa-a1d1-253c9bd7dd8f None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:37.946807 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d11e630-cedf-46aa-a1d1-253c9bd7dd8f None admin] PATH_INFO: `/v3/services` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:37.947642 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d11e630-cedf-46aa-a1d1-253c9bd7dd8f None admin] RBAC: Adding query filter params (type=compute) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:48:37.948101 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d11e630-cedf-46aa-a1d1-253c9bd7dd8f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:37.950033 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d11e630-cedf-46aa-a1d1-253c9bd7dd8f None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:37.953057 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:37.966716 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 461/956] 10.222.0.46 () {56 vars in 1112 bytes} [Thu Jun 23 15:48:37 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:48:37.966716 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d11e630-cedf-46aa-a1d1-253c9bd7dd8f None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:38.043265 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b7b099-fee7-4572-9f51-a02d79116366 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:38.095453 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604bbf35-232e-427e-b8f4-ab6609de23d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:38.097050 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604bbf35-232e-427e-b8f4-ab6609de23d4 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:38.097355 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604bbf35-232e-427e-b8f4-ab6609de23d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:38.097679 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604bbf35-232e-427e-b8f4-ab6609de23d4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:38.100555 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d11e630-cedf-46aa-a1d1-253c9bd7dd8f None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:38.107216 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604bbf35-232e-427e-b8f4-ab6609de23d4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:38.108887 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604bbf35-232e-427e-b8f4-ab6609de23d4 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:38.126527 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-604bbf35-232e-427e-b8f4-ab6609de23d4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 15:48:38.126527 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:48:38.126527 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:48:38.126527 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:48:38.126527 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:48:38.126527 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:48:38.126527 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:48:38.126527 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:48:38.126527 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:48:38.126527 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:48:38.126527 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:48:38.126527 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:48:38.126527 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 15:48:38.126527 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 15:48:38.126527 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 15:48:38.127565 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 15:48:38.127565 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:48:38.127565 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:48:38.127565 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:48:38.127565 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:48:38.127565 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:48:38.127565 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:48:38.127565 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:48:38.127565 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Jun 23 15:48:38.127565 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:48:38.127565 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:48:38.127565 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:48:38.127565 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:48:38.127565 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:48:38.127565 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:48:38.127565 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:48:38.127565 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:48:38.128506 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 15:48:38.128506 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 15:48:38.128506 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 15:48:38.128506 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 15:48:38.128506 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 15:48:38.128506 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 15:48:38.128506 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 15:48:38.128506 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Jun 23 15:48:38.129998 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 496/957] 10.222.0.46 () {56 vars in 1111 bytes} [Thu Jun 23 15:48:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:48:38.135395 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604bbf35-232e-427e-b8f4-ab6609de23d4 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:38.147268 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6619e9b-e202-417c-9241-ab3435a45694 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:38.147947 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6619e9b-e202-417c-9241-ab3435a45694 None admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:38.148264 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6619e9b-e202-417c-9241-ab3435a45694 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:38.148509 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6619e9b-e202-417c-9241-ab3435a45694 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:38.149522 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6619e9b-e202-417c-9241-ab3435a45694 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:38.151080 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6619e9b-e202-417c-9241-ab3435a45694 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:38.175297 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 462/958] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Jun 23 15:48:37 2022] POST /identity/v3/endpoints => generated 349 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 15:48:38.258166 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:38.266240 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604bbf35-232e-427e-b8f4-ab6609de23d4 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:38.314844 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e7f572-5359-44a5-a412-ed81dd9fd8ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:38.315528 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e7f572-5359-44a5-a412-ed81dd9fd8ff None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:38.315811 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e7f572-5359-44a5-a412-ed81dd9fd8ff None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:38.316083 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e7f572-5359-44a5-a412-ed81dd9fd8ff None admin] PATH_INFO: `/v3/projects` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:38.317001 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e7f572-5359-44a5-a412-ed81dd9fd8ff None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:48:38.317318 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e7f572-5359-44a5-a412-ed81dd9fd8ff None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:38.318875 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e7f572-5359-44a5-a412-ed81dd9fd8ff None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:38.333124 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 497/959] 10.222.0.46 () {56 vars in 1148 bytes} [Thu Jun 23 15:48:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:48:38.339590 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6619e9b-e202-417c-9241-ab3435a45694 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:38.361166 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:38.379128 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8970fa-dc9f-4ba6-8ae5-815e8f388ff8 None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:38.379654 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8970fa-dc9f-4ba6-8ae5-815e8f388ff8 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:38.379654 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8970fa-dc9f-4ba6-8ae5-815e8f388ff8 None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:38.381282 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 498/960] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jun 23 15:48:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 15:48:38.386369 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a98df4-de65-4227-a4ea-3776537c10c6 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:38.386653 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a98df4-de65-4227-a4ea-3776537c10c6 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:38.386963 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a98df4-de65-4227-a4ea-3776537c10c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:38.407171 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-25a98df4-de65-4227-a4ea-3776537c10c6 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:48:38.408377 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-25a98df4-de65-4227-a4ea-3776537c10c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:48:38.461267 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6619e9b-e202-417c-9241-ab3435a45694 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:38.495983 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 499/961] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jun 23 15:48:38 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 23 15:48:38.500358 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f7b2d7-6944-42fb-a25c-e57885ff7429 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:38.501493 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f7b2d7-6944-42fb-a25c-e57885ff7429 None admin] REQUEST_METHOD: `PUT` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:38.501971 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f7b2d7-6944-42fb-a25c-e57885ff7429 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:38.502440 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f7b2d7-6944-42fb-a25c-e57885ff7429 None admin] PATH_INFO: `/v3/projects/28ecdf5b474c44098c722f677d2a0b77/users/be48addfaa154555b8d0cb7aac8b32ac/roles/50fafe10bef649b4b87dfab0e16e7915` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:38.504941 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f7b2d7-6944-42fb-a25c-e57885ff7429 None admin] RBAC: Authorizing `identity:create_grant(project_id=28ecdf5b474c44098c722f677d2a0b77, user_id=be48addfaa154555b8d0cb7aac8b32ac, role_id=50fafe10bef649b4b87dfab0e16e7915)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:38.506604 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f7b2d7-6944-42fb-a25c-e57885ff7429 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:38.522014 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a98df4-de65-4227-a4ea-3776537c10c6 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:38.527942 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 463/962] 10.222.0.46 () {58 vars in 1439 bytes} [Thu Jun 23 15:48:38 2022] PUT /identity/v3/projects/28ecdf5b474c44098c722f677d2a0b77/users/be48addfaa154555b8d0cb7aac8b32ac/roles/50fafe10bef649b4b87dfab0e16e7915 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 15:48:38.619710 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:38.640403 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a98df4-de65-4227-a4ea-3776537c10c6 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:38.687661 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ce9b50-6bbc-4cb1-92df-ebf12347238b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:38.688357 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ce9b50-6bbc-4cb1-92df-ebf12347238b None admin] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:38.688640 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ce9b50-6bbc-4cb1-92df-ebf12347238b None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:38.688916 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ce9b50-6bbc-4cb1-92df-ebf12347238b None admin] PATH_INFO: `/v3/users` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:38.689635 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ce9b50-6bbc-4cb1-92df-ebf12347238b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:38.692613 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ce9b50-6bbc-4cb1-92df-ebf12347238b None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:38.708517 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.sql.core [None req-c0ce9b50-6bbc-4cb1-92df-ebf12347238b None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 23 15:48:38.708517 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jun 23 15:48:38.708517 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [parameters: {'user_id': '6d385d93c83b48b48933f1f53866ff6a', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jun 23 15:48:38.708517 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80153) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 23 15:48:38.716548 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-c0ce9b50-6bbc-4cb1-92df-ebf12347238b None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jun 23 15:48:38.716548 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:48:38.716548 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 23 15:48:38.716548 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 23 15:48:38.716548 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 23 15:48:38.716548 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 23 15:48:38.716548 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 23 15:48:38.716548 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application result = self._query(query) Jun 23 15:48:38.716548 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 23 15:48:38.716548 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application conn.query(q) Jun 23 15:48:38.716548 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 23 15:48:38.716548 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 23 15:48:38.716548 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 23 15:48:38.716548 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application result.read() Jun 23 15:48:38.716548 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 23 15:48:38.717449 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 23 15:48:38.717449 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 23 15:48:38.717449 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 23 15:48:38.717449 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 23 15:48:38.717449 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 23 15:48:38.717449 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 23 15:48:38.717449 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 23 15:48:38.717449 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 23 15:48:38.717449 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Jun 23 15:48:38.717449 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 23 15:48:38.717449 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Jun 23 15:48:38.717449 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:48:38.717449 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 23 15:48:38.717449 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 23 15:48:38.717449 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jun 23 15:48:38.717449 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jun 23 15:48:38.717449 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 23 15:48:38.717449 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application next(self.gen) Jun 23 15:48:38.718400 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 23 15:48:38.718400 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application yield resource Jun 23 15:48:38.718400 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 23 15:48:38.718400 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application next(self.gen) Jun 23 15:48:38.718400 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 23 15:48:38.718400 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application self.session.rollback() Jun 23 15:48:38.718400 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 23 15:48:38.718400 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application self.force_reraise() Jun 23 15:48:38.718400 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 23 15:48:38.718400 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise self.value Jun 23 15:48:38.718400 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 23 15:48:38.718400 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 23 15:48:38.718400 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 23 15:48:38.718400 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application session.commit() Jun 23 15:48:38.718400 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 23 15:48:38.718400 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 23 15:48:38.718400 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 23 15:48:38.720775 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application self._prepare_impl() Jun 23 15:48:38.720775 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 23 15:48:38.720775 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application self.session.flush() Jun 23 15:48:38.720775 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 23 15:48:38.720775 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application self._flush(objects) Jun 23 15:48:38.720775 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 23 15:48:38.720775 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 23 15:48:38.720775 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 23 15:48:38.720775 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application compat.raise_( Jun 23 15:48:38.720775 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 23 15:48:38.720775 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception Jun 23 15:48:38.720775 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 23 15:48:38.720775 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application flush_context.execute() Jun 23 15:48:38.720775 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 23 15:48:38.720775 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rec.execute(self) Jun 23 15:48:38.720775 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 23 15:48:38.720775 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 23 15:48:38.722926 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 23 15:48:38.722926 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 23 15:48:38.722926 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jun 23 15:48:38.722926 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application result = connection._execute_20( Jun 23 15:48:38.722926 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 23 15:48:38.722926 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 23 15:48:38.722926 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 23 15:48:38.722926 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 23 15:48:38.722926 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 23 15:48:38.722926 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 23 15:48:38.722926 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 23 15:48:38.722926 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 23 15:48:38.722926 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 23 15:48:38.722926 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 23 15:48:38.722926 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 23 15:48:38.722926 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception Jun 23 15:48:38.724563 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 23 15:48:38.724563 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 23 15:48:38.724563 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 23 15:48:38.724563 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 23 15:48:38.724563 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 23 15:48:38.724563 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application result = self._query(query) Jun 23 15:48:38.724563 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 23 15:48:38.724563 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application conn.query(q) Jun 23 15:48:38.724563 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 23 15:48:38.724563 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 23 15:48:38.724563 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 23 15:48:38.724563 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application result.read() Jun 23 15:48:38.724563 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 23 15:48:38.724563 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 23 15:48:38.724563 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 23 15:48:38.724563 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 23 15:48:38.725662 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 23 15:48:38.725662 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 23 15:48:38.725662 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 23 15:48:38.725662 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 23 15:48:38.725662 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 23 15:48:38.725662 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jun 23 15:48:38.725662 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [parameters: {'user_id': '6d385d93c83b48b48933f1f53866ff6a', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jun 23 15:48:38.725662 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 23 15:48:38.725662 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Jun 23 15:48:38.725662 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 15:48:38.725662 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Jun 23 15:48:38.725662 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:48:38.725662 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:48:38.725662 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:48:38.725662 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:48:38.729789 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:48:38.729789 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:48:38.729789 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:48:38.729789 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:48:38.729789 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:48:38.729789 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:48:38.729789 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:48:38.729789 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jun 23 15:48:38.729789 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jun 23 15:48:38.729789 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:48:38.729789 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:48:38.729789 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 15:48:38.729789 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 15:48:38.729789 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 15:48:38.729789 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 15:48:38.729789 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jun 23 15:48:38.729789 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jun 23 15:48:38.731562 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jun 23 15:48:38.731562 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jun 23 15:48:38.731562 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 23 15:48:38.731562 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 23 15:48:38.731562 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jun 23 15:48:38.731562 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Jun 23 15:48:38.731562 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 500/963] 10.222.0.46 () {60 vars in 1131 bytes} [Thu Jun 23 15:48:38 2022] POST /identity/v3/users => generated 160 bytes in 197 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:48:38.732234 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f7b2d7-6944-42fb-a25c-e57885ff7429 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:38.815940 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:38.839932 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f7b2d7-6944-42fb-a25c-e57885ff7429 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:38.879501 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792795c5-ea93-4c58-8139-848df53937b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:38.880239 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792795c5-ea93-4c58-8139-848df53937b8 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:38.880518 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792795c5-ea93-4c58-8139-848df53937b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:38.880837 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792795c5-ea93-4c58-8139-848df53937b8 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:38.888791 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792795c5-ea93-4c58-8139-848df53937b8 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:38.890243 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792795c5-ea93-4c58-8139-848df53937b8 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:38.904880 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-792795c5-ea93-4c58-8139-848df53937b8 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 23 15:48:38.904880 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:48:38.904880 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:48:38.904880 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:48:38.904880 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:48:38.904880 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:48:38.904880 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:48:38.904880 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:48:38.904880 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:48:38.904880 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:48:38.904880 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:48:38.904880 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:48:38.904880 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 15:48:38.904880 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 15:48:38.904880 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 15:48:38.904880 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 15:48:38.905850 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:48:38.905850 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:48:38.905850 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 15:48:38.905850 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 15:48:38.905850 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 15:48:38.905850 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 15:48:38.905850 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:48:38.905850 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:48:38.905850 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:48:38.905850 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:48:38.905850 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:48:38.905850 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Jun 23 15:48:38.905850 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:48:38.905850 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:48:38.905850 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:48:38.905850 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:48:38.905850 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:48:38.905850 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:48:38.906827 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:48:38.906827 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:48:38.906827 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 15:48:38.906827 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 15:48:38.906827 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 15:48:38.906827 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 15:48:38.906827 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 15:48:38.906827 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 15:48:38.906827 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 23 15:48:38.906827 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Jun 23 15:48:38.906827 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 464/964] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Jun 23 15:48:38 2022] GET /identity/v3/users/cinder => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:48:38.912625 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ce9b50-6bbc-4cb1-92df-ebf12347238b None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:39.031739 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ce9b50-6bbc-4cb1-92df-ebf12347238b None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:39.073217 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094a1720-f12f-4bc9-8581-7328b7296785 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:39.073903 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094a1720-f12f-4bc9-8581-7328b7296785 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:39.074190 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094a1720-f12f-4bc9-8581-7328b7296785 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:39.074467 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094a1720-f12f-4bc9-8581-7328b7296785 None admin] PATH_INFO: `/v3/users` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:39.075315 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094a1720-f12f-4bc9-8581-7328b7296785 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:48:39.075624 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094a1720-f12f-4bc9-8581-7328b7296785 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:39.077277 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094a1720-f12f-4bc9-8581-7328b7296785 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:39.118594 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 501/965] 10.222.0.46 () {56 vars in 1101 bytes} [Thu Jun 23 15:48:38 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:48:40.132427 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc47fcf5-6420-44f7-83b0-d69f6fddc7b1 None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:40.132978 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc47fcf5-6420-44f7-83b0-d69f6fddc7b1 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:40.132978 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc47fcf5-6420-44f7-83b0-d69f6fddc7b1 None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:40.134497 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 465/966] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jun 23 15:48:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 15:48:40.139795 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd38f0bf-192a-484e-bcb6-743e2b1a1263 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:40.140254 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd38f0bf-192a-484e-bcb6-743e2b1a1263 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:40.140429 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd38f0bf-192a-484e-bcb6-743e2b1a1263 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:40.145807 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:40.158992 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-bd38f0bf-192a-484e-bcb6-743e2b1a1263 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:48:40.160555 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-bd38f0bf-192a-484e-bcb6-743e2b1a1263 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:48:40.234727 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 502/967] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jun 23 15:48:40 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 15:48:40.258227 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc47fcf5-6420-44f7-83b0-d69f6fddc7b1 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:40.369565 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc47fcf5-6420-44f7-83b0-d69f6fddc7b1 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:40.389207 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661268b0-d03a-4b8a-9e3a-2bff3cbf8a88 None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:40.389499 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661268b0-d03a-4b8a-9e3a-2bff3cbf8a88 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:40.389748 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661268b0-d03a-4b8a-9e3a-2bff3cbf8a88 None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:40.391171 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 503/968] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jun 23 15:48:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 15:48:40.396826 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef02d7c5-ad17-4df8-9b67-5ee265d2fcfe None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:40.397360 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef02d7c5-ad17-4df8-9b67-5ee265d2fcfe None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:40.397877 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef02d7c5-ad17-4df8-9b67-5ee265d2fcfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:40.404845 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4a9678-43ab-45fb-b356-34555e5918d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:40.405529 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4a9678-43ab-45fb-b356-34555e5918d4 None admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:40.405814 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4a9678-43ab-45fb-b356-34555e5918d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:40.406092 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4a9678-43ab-45fb-b356-34555e5918d4 None admin] PATH_INFO: `/v3/services` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:40.407059 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4a9678-43ab-45fb-b356-34555e5918d4 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:40.408903 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4a9678-43ab-45fb-b356-34555e5918d4 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:40.417103 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-ef02d7c5-ad17-4df8-9b67-5ee265d2fcfe None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:48:40.417715 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:40.418515 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-ef02d7c5-ad17-4df8-9b67-5ee265d2fcfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:48:40.431134 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 466/969] 10.222.0.46 () {60 vars in 1141 bytes} [Thu Jun 23 15:48:40 2022] POST /identity/v3/services => generated 242 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 15:48:40.511743 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 504/970] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jun 23 15:48:40 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 23 15:48:40.535774 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4a9678-43ab-45fb-b356-34555e5918d4 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:40.621204 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d8d935-8c7c-4ff9-ae6b-ab626608dcf6 None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:40.621204 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d8d935-8c7c-4ff9-ae6b-ab626608dcf6 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:40.621487 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d8d935-8c7c-4ff9-ae6b-ab626608dcf6 None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:40.622834 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 505/971] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jun 23 15:48:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 15:48:40.628035 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19086e29-bd6a-485d-a5e3-de454c26b575 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:40.628325 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19086e29-bd6a-485d-a5e3-de454c26b575 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:40.628602 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19086e29-bd6a-485d-a5e3-de454c26b575 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:40.645708 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4a9678-43ab-45fb-b356-34555e5918d4 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:40.645891 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-19086e29-bd6a-485d-a5e3-de454c26b575 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:48:40.646939 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-19086e29-bd6a-485d-a5e3-de454c26b575 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:48:40.682790 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3b5a0b-30ee-419d-80d7-8226c5fbe996 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:40.684088 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3b5a0b-30ee-419d-80d7-8226c5fbe996 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:40.684559 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3b5a0b-30ee-419d-80d7-8226c5fbe996 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:40.685043 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3b5a0b-30ee-419d-80d7-8226c5fbe996 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:40.685709 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3b5a0b-30ee-419d-80d7-8226c5fbe996 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:40.687125 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3b5a0b-30ee-419d-80d7-8226c5fbe996 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:40.699516 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-5b3b5a0b-30ee-419d-80d7-8226c5fbe996 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 23 15:48:40.699516 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:48:40.699516 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 15:48:40.699516 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 15:48:40.699516 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 15:48:40.699516 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 15:48:40.699516 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 15:48:40.699516 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 15:48:40.699516 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 23 15:48:40.699516 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Jun 23 15:48:40.699516 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 15:48:40.699516 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Jun 23 15:48:40.699516 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:48:40.699516 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:48:40.699516 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:48:40.699516 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:48:40.699516 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:48:40.700677 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:48:40.700677 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:48:40.700677 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:48:40.700677 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:48:40.700677 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:48:40.700677 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:48:40.700677 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 15:48:40.700677 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 15:48:40.700677 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 15:48:40.700677 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 15:48:40.700677 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:48:40.700677 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:48:40.700677 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:48:40.700677 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:48:40.700677 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:48:40.700677 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:48:40.702217 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:48:40.702217 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Jun 23 15:48:40.702217 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:48:40.702217 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:48:40.702217 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:48:40.702217 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:48:40.702217 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:48:40.702217 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:48:40.702217 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:48:40.702217 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:48:40.702217 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 15:48:40.702217 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 15:48:40.702217 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 23 15:48:40.702217 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Jun 23 15:48:40.702217 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 467/972] 10.222.0.46 () {56 vars in 1126 bytes} [Thu Jun 23 15:48:40 2022] GET /identity/v3/services/object-store => generated 93 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 15:48:40.707028 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3b5a0b-30ee-419d-80d7-8226c5fbe996 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:40.714800 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:40.730968 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 506/973] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jun 23 15:48:40 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 23 15:48:40.755556 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19086e29-bd6a-485d-a5e3-de454c26b575 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:40.777447 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:40.825009 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3b5a0b-30ee-419d-80d7-8226c5fbe996 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:40.860974 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79289ac-3991-4e6d-ac5a-22f30218382a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:40.861655 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79289ac-3991-4e6d-ac5a-22f30218382a None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:40.861945 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79289ac-3991-4e6d-ac5a-22f30218382a None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:40.862227 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79289ac-3991-4e6d-ac5a-22f30218382a None admin] PATH_INFO: `/v3/services` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:40.862996 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79289ac-3991-4e6d-ac5a-22f30218382a None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:48:40.863315 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79289ac-3991-4e6d-ac5a-22f30218382a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:40.864705 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79289ac-3991-4e6d-ac5a-22f30218382a None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:40.874034 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 468/974] 10.222.0.46 () {56 vars in 1122 bytes} [Thu Jun 23 15:48:40 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 15:48:40.874186 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19086e29-bd6a-485d-a5e3-de454c26b575 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:40.875076 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79289ac-3991-4e6d-ac5a-22f30218382a None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:40.909745 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c96627b-fb69-426d-bdd6-072a29eb5d34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:40.911056 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c96627b-fb69-426d-bdd6-072a29eb5d34 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:40.911335 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c96627b-fb69-426d-bdd6-072a29eb5d34 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:40.911606 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c96627b-fb69-426d-bdd6-072a29eb5d34 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:40.922739 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c96627b-fb69-426d-bdd6-072a29eb5d34 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 15:48:40.930723 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-9c96627b-fb69-426d-bdd6-072a29eb5d34 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 23 15:48:40.930723 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:48:40.930723 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:48:40.930723 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:48:40.930723 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:48:40.930723 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:48:40.930723 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:48:40.930723 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:48:40.930723 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:48:40.930723 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:48:40.930723 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:48:40.930723 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:48:40.930723 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 15:48:40.930723 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 15:48:40.930723 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 15:48:40.930723 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 15:48:40.931696 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 15:48:40.931696 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 15:48:40.931696 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 15:48:40.931696 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 15:48:40.931696 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:48:40.931696 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:48:40.931696 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:48:40.931696 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:48:40.931696 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:48:40.931696 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:48:40.931696 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:48:40.931696 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Jun 23 15:48:40.931696 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:48:40.931696 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:48:40.931696 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:48:40.931696 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:48:40.932540 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:48:40.932540 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:48:40.932540 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:48:40.932540 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:48:40.932540 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 15:48:40.932540 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 15:48:40.932540 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 15:48:40.932540 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 15:48:40.932540 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 15:48:40.932540 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 15:48:40.932540 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 23 15:48:40.932540 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Jun 23 15:48:40.932540 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 507/975] 10.222.0.46 () {56 vars in 1120 bytes} [Thu Jun 23 15:48:40 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 15:48:40.937606 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c96627b-fb69-426d-bdd6-072a29eb5d34 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:40.981296 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79289ac-3991-4e6d-ac5a-22f30218382a None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:40.989368 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:41.007637 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:41.018401 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e19210-f3d0-4402-8a15-62fc1ed4b8d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:41.019860 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e19210-f3d0-4402-8a15-62fc1ed4b8d7 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:41.020303 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e19210-f3d0-4402-8a15-62fc1ed4b8d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:41.020577 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e19210-f3d0-4402-8a15-62fc1ed4b8d7 None admin] PATH_INFO: `/v3/services` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:41.021290 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e19210-f3d0-4402-8a15-62fc1ed4b8d7 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:48:41.021598 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e19210-f3d0-4402-8a15-62fc1ed4b8d7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:41.022982 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e19210-f3d0-4402-8a15-62fc1ed4b8d7 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:41.028244 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 469/976] 10.222.0.46 () {56 vars in 1122 bytes} [Thu Jun 23 15:48:40 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 15:48:41.033856 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e19210-f3d0-4402-8a15-62fc1ed4b8d7 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:41.046152 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c96627b-fb69-426d-bdd6-072a29eb5d34 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:41.080752 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967c1b36-d7e8-4d05-8461-662d4f309571 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:41.081433 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967c1b36-d7e8-4d05-8461-662d4f309571 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:41.082240 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967c1b36-d7e8-4d05-8461-662d4f309571 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:41.082535 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967c1b36-d7e8-4d05-8461-662d4f309571 None admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:41.083320 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967c1b36-d7e8-4d05-8461-662d4f309571 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:48:41.083634 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967c1b36-d7e8-4d05-8461-662d4f309571 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:41.085028 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967c1b36-d7e8-4d05-8461-662d4f309571 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:41.094745 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 508/977] 10.222.0.46 () {56 vars in 1115 bytes} [Thu Jun 23 15:48:40 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 15:48:41.099615 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967c1b36-d7e8-4d05-8461-662d4f309571 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:41.143843 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e19210-f3d0-4402-8a15-62fc1ed4b8d7 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:41.179998 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da53d5e-842e-4ff9-9700-040fc62ef097 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:41.180579 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da53d5e-842e-4ff9-9700-040fc62ef097 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:41.180862 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da53d5e-842e-4ff9-9700-040fc62ef097 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:41.181143 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da53d5e-842e-4ff9-9700-040fc62ef097 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:41.181938 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da53d5e-842e-4ff9-9700-040fc62ef097 None admin] RBAC: Adding query filter params (interface=admin, service_id=fe53c3c741d14b4c973ff1a6e00fde97, region_id=RegionOne) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:48:41.182249 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da53d5e-842e-4ff9-9700-040fc62ef097 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:41.183653 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da53d5e-842e-4ff9-9700-040fc62ef097 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:41.188866 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 470/978] 10.222.0.46 () {56 vars in 1249 bytes} [Thu Jun 23 15:48:41 2022] GET /identity/v3/endpoints?service_id=fe53c3c741d14b4c973ff1a6e00fde97&interface=admin®ion_id=RegionOne => generated 193 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 15:48:41.194802 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da53d5e-842e-4ff9-9700-040fc62ef097 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:41.212477 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967c1b36-d7e8-4d05-8461-662d4f309571 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:41.225525 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:41.256121 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:41.262403 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82a249e-6844-488e-845b-801946fbc98e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:41.263392 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82a249e-6844-488e-845b-801946fbc98e None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:41.263672 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82a249e-6844-488e-845b-801946fbc98e None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:41.264249 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82a249e-6844-488e-845b-801946fbc98e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:41.278706 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82a249e-6844-488e-845b-801946fbc98e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:41.281933 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82a249e-6844-488e-845b-801946fbc98e None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:41.304182 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-b82a249e-6844-488e-845b-801946fbc98e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 15:48:41.304182 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:48:41.304182 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:48:41.304182 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:48:41.304182 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:48:41.304182 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:48:41.304182 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:48:41.304182 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:48:41.304182 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:48:41.304182 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:48:41.304182 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:48:41.304182 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:48:41.304182 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 15:48:41.304182 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 15:48:41.304182 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 15:48:41.305236 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 15:48:41.305236 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:48:41.305236 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:48:41.305236 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:48:41.305236 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:48:41.305236 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:48:41.305236 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:48:41.305236 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:48:41.305236 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Jun 23 15:48:41.305236 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:48:41.305236 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:48:41.305236 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:48:41.305236 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:48:41.305236 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:48:41.305236 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:48:41.305236 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:48:41.305236 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:48:41.306119 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 15:48:41.306119 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 15:48:41.306119 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 15:48:41.306119 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Jun 23 15:48:41.306119 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 509/979] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Jun 23 15:48:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 15:48:41.311211 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82a249e-6844-488e-845b-801946fbc98e None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:41.312485 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da53d5e-842e-4ff9-9700-040fc62ef097 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:41.350850 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fc1b57-0405-4d8f-99ce-027b4474c7a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:41.351889 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fc1b57-0405-4d8f-99ce-027b4474c7a1 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:41.352573 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fc1b57-0405-4d8f-99ce-027b4474c7a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:41.353057 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fc1b57-0405-4d8f-99ce-027b4474c7a1 None admin] PATH_INFO: `/v3/services` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:41.354648 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fc1b57-0405-4d8f-99ce-027b4474c7a1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:41.356200 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fc1b57-0405-4d8f-99ce-027b4474c7a1 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:41.364222 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 471/980] 10.222.0.46 () {56 vars in 1087 bytes} [Thu Jun 23 15:48:41 2022] GET /identity/v3/services => generated 1727 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 15:48:41.426338 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82a249e-6844-488e-845b-801946fbc98e None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:41.464352 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af5ee7a-2ac0-4f9e-bc13-d666ba9b4e89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:41.464957 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af5ee7a-2ac0-4f9e-bc13-d666ba9b4e89 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:41.465243 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af5ee7a-2ac0-4f9e-bc13-d666ba9b4e89 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:41.465522 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af5ee7a-2ac0-4f9e-bc13-d666ba9b4e89 None admin] PATH_INFO: `/v3/domains` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:41.466268 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af5ee7a-2ac0-4f9e-bc13-d666ba9b4e89 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:48:41.466577 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af5ee7a-2ac0-4f9e-bc13-d666ba9b4e89 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:41.468020 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af5ee7a-2ac0-4f9e-bc13-d666ba9b4e89 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:41.481582 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 510/981] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Jun 23 15:48:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 15:48:41.487252 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fc1b57-0405-4d8f-99ce-027b4474c7a1 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:41.601691 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fc1b57-0405-4d8f-99ce-027b4474c7a1 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:41.619974 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:41.638046 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7136e82e-fecd-4520-a7b4-1ee8166e9bbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:41.638905 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7136e82e-fecd-4520-a7b4-1ee8166e9bbc None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:41.639189 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7136e82e-fecd-4520-a7b4-1ee8166e9bbc None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:41.639461 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7136e82e-fecd-4520-a7b4-1ee8166e9bbc None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:41.648014 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7136e82e-fecd-4520-a7b4-1ee8166e9bbc None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:41.649817 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7136e82e-fecd-4520-a7b4-1ee8166e9bbc None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:41.664333 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-7136e82e-fecd-4520-a7b4-1ee8166e9bbc None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 23 15:48:41.664333 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:48:41.664333 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:48:41.664333 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:48:41.664333 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:48:41.664333 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:48:41.664333 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:48:41.664333 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:48:41.664333 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:48:41.664333 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:48:41.664333 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:48:41.664333 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:48:41.664333 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 15:48:41.664333 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 15:48:41.664333 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 15:48:41.664333 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 15:48:41.665291 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:48:41.665291 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:48:41.665291 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 15:48:41.665291 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 15:48:41.665291 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 15:48:41.665291 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 15:48:41.665291 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:48:41.665291 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:48:41.665291 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:48:41.665291 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:48:41.665291 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:48:41.665291 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Jun 23 15:48:41.665291 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:48:41.665291 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:48:41.665291 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:48:41.665291 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:48:41.665291 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:48:41.665291 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:48:41.666234 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:48:41.666234 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:48:41.666234 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 15:48:41.666234 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 15:48:41.666234 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 15:48:41.666234 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 15:48:41.666234 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 15:48:41.666234 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 15:48:41.666234 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 23 15:48:41.666234 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Jun 23 15:48:41.666234 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 472/982] 10.222.0.46 () {56 vars in 1117 bytes} [Thu Jun 23 15:48:41 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:48:41.672537 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af5ee7a-2ac0-4f9e-bc13-d666ba9b4e89 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:41.698340 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:41.778340 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af5ee7a-2ac0-4f9e-bc13-d666ba9b4e89 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:41.814178 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140d0876-618e-4462-a283-a011ff91d4c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:41.814880 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140d0876-618e-4462-a283-a011ff91d4c6 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:41.815164 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140d0876-618e-4462-a283-a011ff91d4c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:41.815436 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140d0876-618e-4462-a283-a011ff91d4c6 None admin] PATH_INFO: `/v3/users` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:41.816298 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140d0876-618e-4462-a283-a011ff91d4c6 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:48:41.816606 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140d0876-618e-4462-a283-a011ff91d4c6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:41.818036 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140d0876-618e-4462-a283-a011ff91d4c6 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:41.837529 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 511/983] 10.222.0.46 () {56 vars in 1149 bytes} [Thu Jun 23 15:48:41 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 15:48:41.843372 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7136e82e-fecd-4520-a7b4-1ee8166e9bbc None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:41.943022 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7136e82e-fecd-4520-a7b4-1ee8166e9bbc None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:41.976017 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891da6b9-7264-4306-a4f6-8ca43db8d5ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:41.976669 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891da6b9-7264-4306-a4f6-8ca43db8d5ea None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:41.976949 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891da6b9-7264-4306-a4f6-8ca43db8d5ea None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:41.977222 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891da6b9-7264-4306-a4f6-8ca43db8d5ea None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:42.023597 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:42.046585 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891da6b9-7264-4306-a4f6-8ca43db8d5ea None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:42.047987 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891da6b9-7264-4306-a4f6-8ca43db8d5ea None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:42.146640 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 473/984] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Jun 23 15:48:41 2022] GET /identity/v3/auth/tokens => generated 1560 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 23 15:48:42.152458 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140d0876-618e-4462-a283-a011ff91d4c6 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:42.232802 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:42.254471 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140d0876-618e-4462-a283-a011ff91d4c6 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:42.288583 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60ffa4b-053b-4060-be04-826ac4a7af25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:42.289380 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60ffa4b-053b-4060-be04-826ac4a7af25 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:42.289654 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60ffa4b-053b-4060-be04-826ac4a7af25 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:42.289924 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60ffa4b-053b-4060-be04-826ac4a7af25 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:42.300379 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60ffa4b-053b-4060-be04-826ac4a7af25 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:42.302291 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60ffa4b-053b-4060-be04-826ac4a7af25 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:42.320870 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-d60ffa4b-053b-4060-be04-826ac4a7af25 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 15:48:42.320870 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:48:42.320870 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:48:42.320870 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:48:42.320870 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:48:42.320870 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:48:42.320870 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:48:42.320870 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:48:42.320870 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:48:42.320870 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:48:42.320870 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:48:42.320870 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:48:42.320870 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 15:48:42.320870 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 15:48:42.320870 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 15:48:42.321742 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 15:48:42.321742 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:48:42.321742 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:48:42.321742 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:48:42.321742 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:48:42.321742 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:48:42.321742 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:48:42.321742 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:48:42.321742 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Jun 23 15:48:42.321742 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:48:42.321742 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:48:42.321742 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:48:42.321742 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:48:42.321742 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:48:42.321742 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:48:42.321742 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:48:42.321742 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:48:42.323156 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 15:48:42.323156 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 15:48:42.323156 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 15:48:42.323156 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Jun 23 15:48:42.326638 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 512/985] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Jun 23 15:48:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:48:42.327941 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891da6b9-7264-4306-a4f6-8ca43db8d5ea None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:42.404694 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:42.429029 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891da6b9-7264-4306-a4f6-8ca43db8d5ea None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:42.464280 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731b2f9f-36f7-4ba1-b5f3-649a43456c00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:42.464957 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731b2f9f-36f7-4ba1-b5f3-649a43456c00 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:42.465245 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731b2f9f-36f7-4ba1-b5f3-649a43456c00 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:42.465522 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731b2f9f-36f7-4ba1-b5f3-649a43456c00 None admin] PATH_INFO: `/v3/domains` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:42.466251 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731b2f9f-36f7-4ba1-b5f3-649a43456c00 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:48:42.466559 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731b2f9f-36f7-4ba1-b5f3-649a43456c00 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:42.467961 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731b2f9f-36f7-4ba1-b5f3-649a43456c00 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:42.480893 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 474/986] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Jun 23 15:48:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:48:42.486626 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60ffa4b-053b-4060-be04-826ac4a7af25 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:42.512093 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828db763-0a64-4c3f-a2df-e9375feb55fd None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:42.512385 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828db763-0a64-4c3f-a2df-e9375feb55fd None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:42.512660 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828db763-0a64-4c3f-a2df-e9375feb55fd None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:42.514033 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 475/987] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jun 23 15:48:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 15:48:42.518782 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e548f0-a1a8-4030-aaf3-1e49a1ca2763 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:42.519078 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e548f0-a1a8-4030-aaf3-1e49a1ca2763 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:42.519434 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e548f0-a1a8-4030-aaf3-1e49a1ca2763 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:42.536824 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-28e548f0-a1a8-4030-aaf3-1e49a1ca2763 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:48:42.538049 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-28e548f0-a1a8-4030-aaf3-1e49a1ca2763 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:48:42.605123 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60ffa4b-053b-4060-be04-826ac4a7af25 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:42.618734 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 476/988] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jun 23 15:48:42 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 23 15:48:42.637935 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:42.640255 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e548f0-a1a8-4030-aaf3-1e49a1ca2763 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:42.646479 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d711e3db-989e-4820-9c26-1e15dcb5b4aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:42.647320 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d711e3db-989e-4820-9c26-1e15dcb5b4aa None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:42.647678 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d711e3db-989e-4820-9c26-1e15dcb5b4aa None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:42.647883 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d711e3db-989e-4820-9c26-1e15dcb5b4aa None admin] PATH_INFO: `/v3/projects/service` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:42.656713 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d711e3db-989e-4820-9c26-1e15dcb5b4aa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:42.658163 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d711e3db-989e-4820-9c26-1e15dcb5b4aa None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:42.679321 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-d711e3db-989e-4820-9c26-1e15dcb5b4aa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 15:48:42.679321 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:48:42.679321 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:48:42.679321 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:48:42.679321 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:48:42.679321 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:48:42.679321 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:48:42.679321 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:48:42.679321 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:48:42.679321 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:48:42.679321 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:48:42.679321 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:48:42.679321 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 15:48:42.679321 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 15:48:42.679321 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 15:48:42.680234 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 15:48:42.680234 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:48:42.680234 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:48:42.680234 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:48:42.680234 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:48:42.680234 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:48:42.680234 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:48:42.680234 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:48:42.680234 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Jun 23 15:48:42.680234 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:48:42.680234 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:48:42.680234 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:48:42.680234 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:48:42.680234 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:48:42.680234 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:48:42.680234 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:48:42.680234 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:48:42.686324 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 15:48:42.686324 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 15:48:42.686324 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 15:48:42.686324 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 15:48:42.686324 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 15:48:42.686324 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 15:48:42.686324 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 15:48:42.686324 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Jun 23 15:48:42.686324 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 513/989] 10.222.0.46 () {56 vars in 1111 bytes} [Thu Jun 23 15:48:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:48:42.686861 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d711e3db-989e-4820-9c26-1e15dcb5b4aa None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:42.710841 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:42.750451 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e548f0-a1a8-4030-aaf3-1e49a1ca2763 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:42.786039 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c351534-9255-4969-a367-81263b79382a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:42.786874 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c351534-9255-4969-a367-81263b79382a None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:42.787163 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c351534-9255-4969-a367-81263b79382a None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:42.787442 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c351534-9255-4969-a367-81263b79382a None admin] PATH_INFO: `/v3/services/compute` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:42.788400 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c351534-9255-4969-a367-81263b79382a None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:42.789459 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c351534-9255-4969-a367-81263b79382a None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:42.801438 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d711e3db-989e-4820-9c26-1e15dcb5b4aa None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:42.801802 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-2c351534-9255-4969-a367-81263b79382a None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 23 15:48:42.801802 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:48:42.801802 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 15:48:42.801802 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 15:48:42.801802 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 15:48:42.801802 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 15:48:42.801802 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 15:48:42.801802 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 15:48:42.801802 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 23 15:48:42.801802 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Jun 23 15:48:42.801802 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 15:48:42.801802 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Jun 23 15:48:42.801802 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:48:42.801802 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:48:42.801802 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:48:42.801802 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:48:42.801802 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:48:42.802944 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:48:42.802944 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:48:42.802944 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:48:42.802944 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:48:42.802944 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:48:42.802944 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:48:42.802944 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 15:48:42.802944 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 15:48:42.802944 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 15:48:42.802944 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 15:48:42.802944 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:48:42.802944 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:48:42.802944 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:48:42.802944 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:48:42.802944 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:48:42.802944 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:48:42.803875 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:48:42.803875 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Jun 23 15:48:42.803875 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:48:42.803875 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:48:42.803875 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:48:42.803875 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:48:42.803875 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:48:42.803875 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:48:42.803875 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:48:42.803875 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:48:42.803875 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 15:48:42.803875 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 15:48:42.803875 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 23 15:48:42.803875 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Jun 23 15:48:42.803875 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 477/990] 10.222.0.46 () {56 vars in 1111 bytes} [Thu Jun 23 15:48:42 2022] GET /identity/v3/services/compute => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 15:48:42.808692 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c351534-9255-4969-a367-81263b79382a None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:42.839864 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fec0882-4531-4bc1-a51d-b8ce3868a7f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:42.840532 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fec0882-4531-4bc1-a51d-b8ce3868a7f6 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:42.840819 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fec0882-4531-4bc1-a51d-b8ce3868a7f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:42.841281 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fec0882-4531-4bc1-a51d-b8ce3868a7f6 None admin] PATH_INFO: `/v3/projects` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:42.841883 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fec0882-4531-4bc1-a51d-b8ce3868a7f6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:48:42.842193 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fec0882-4531-4bc1-a51d-b8ce3868a7f6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:42.843660 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fec0882-4531-4bc1-a51d-b8ce3868a7f6 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:42.857664 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 514/991] 10.222.0.46 () {56 vars in 1148 bytes} [Thu Jun 23 15:48:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:48:42.863879 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fec0882-4531-4bc1-a51d-b8ce3868a7f6 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:42.914979 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c351534-9255-4969-a367-81263b79382a None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:42.921180 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:42.947851 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:42.954166 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaeea8aa-7254-4fe4-b7dd-1ea7fa247ac8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:42.954872 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaeea8aa-7254-4fe4-b7dd-1ea7fa247ac8 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:42.955158 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaeea8aa-7254-4fe4-b7dd-1ea7fa247ac8 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:42.955436 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaeea8aa-7254-4fe4-b7dd-1ea7fa247ac8 None admin] PATH_INFO: `/v3/services` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:42.956140 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaeea8aa-7254-4fe4-b7dd-1ea7fa247ac8 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:48:42.956448 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaeea8aa-7254-4fe4-b7dd-1ea7fa247ac8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:42.957833 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaeea8aa-7254-4fe4-b7dd-1ea7fa247ac8 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:42.963127 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 478/992] 10.222.0.46 () {56 vars in 1112 bytes} [Thu Jun 23 15:48:42 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 15:48:42.968340 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaeea8aa-7254-4fe4-b7dd-1ea7fa247ac8 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:42.973085 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fec0882-4531-4bc1-a51d-b8ce3868a7f6 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:43.009554 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6082e5-d27d-499a-a7a4-652b02042960 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:43.010307 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6082e5-d27d-499a-a7a4-652b02042960 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:43.010655 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6082e5-d27d-499a-a7a4-652b02042960 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:43.011032 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6082e5-d27d-499a-a7a4-652b02042960 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:43.011880 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6082e5-d27d-499a-a7a4-652b02042960 None admin] RBAC: Adding query filter params (role.id=50fafe10bef649b4b87dfab0e16e7915, scope.project.id=28ecdf5b474c44098c722f677d2a0b77, user.id=be48addfaa154555b8d0cb7aac8b32ac) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:48:43.012194 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6082e5-d27d-499a-a7a4-652b02042960 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:43.013630 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6082e5-d27d-499a-a7a4-652b02042960 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:43.020756 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 515/993] 10.222.0.46 () {56 vars in 1374 bytes} [Thu Jun 23 15:48:42 2022] GET /identity/v3/role_assignments?user.id=be48addfaa154555b8d0cb7aac8b32ac&scope.project.id=28ecdf5b474c44098c722f677d2a0b77&role.id=50fafe10bef649b4b87dfab0e16e7915 => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 15:48:43.078617 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaeea8aa-7254-4fe4-b7dd-1ea7fa247ac8 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:43.117376 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7984df-91b5-4fac-aa7b-60bbe0925aef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:43.118131 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7984df-91b5-4fac-aa7b-60bbe0925aef None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:43.118510 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7984df-91b5-4fac-aa7b-60bbe0925aef None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:43.118936 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7984df-91b5-4fac-aa7b-60bbe0925aef None admin] PATH_INFO: `/v3/services` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:43.119762 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7984df-91b5-4fac-aa7b-60bbe0925aef None admin] RBAC: Adding query filter params (type=compute) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:48:43.120166 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7984df-91b5-4fac-aa7b-60bbe0925aef None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:43.121649 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7984df-91b5-4fac-aa7b-60bbe0925aef None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:43.132226 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 479/994] 10.222.0.46 () {56 vars in 1112 bytes} [Thu Jun 23 15:48:42 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 15:48:43.133995 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6082e5-d27d-499a-a7a4-652b02042960 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:43.242496 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6082e5-d27d-499a-a7a4-652b02042960 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:43.259155 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:43.279032 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62c8017-e711-4ffc-85fd-6b4b772e4afb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:43.279706 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62c8017-e711-4ffc-85fd-6b4b772e4afb None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:43.279989 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62c8017-e711-4ffc-85fd-6b4b772e4afb None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:43.280323 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62c8017-e711-4ffc-85fd-6b4b772e4afb None admin] PATH_INFO: `/v3/endpoints` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:43.281157 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62c8017-e711-4ffc-85fd-6b4b772e4afb None admin] RBAC: Adding query filter params (interface=public, service_id=0fdd7c3c307e45d1a121284e1279c478, region_id=RegionOne) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:48:43.281475 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62c8017-e711-4ffc-85fd-6b4b772e4afb None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:43.282886 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62c8017-e711-4ffc-85fd-6b4b772e4afb None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:43.294777 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 516/995] 10.222.0.46 () {56 vars in 1251 bytes} [Thu Jun 23 15:48:43 2022] GET /identity/v3/endpoints?service_id=0fdd7c3c307e45d1a121284e1279c478&interface=public®ion_id=RegionOne => generated 194 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:48:43.296444 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7984df-91b5-4fac-aa7b-60bbe0925aef None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:43.359853 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:43.404147 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7984df-91b5-4fac-aa7b-60bbe0925aef None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:43.436013 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6414d61e-b831-4522-9ce6-76ccad2e2eee None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:43.436314 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6414d61e-b831-4522-9ce6-76ccad2e2eee None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:43.436570 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6414d61e-b831-4522-9ce6-76ccad2e2eee None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:43.438071 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 517/996] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jun 23 15:48:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 15:48:43.438185 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15036e7-0bfb-4788-9dac-456b4c946bcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:43.438768 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15036e7-0bfb-4788-9dac-456b4c946bcd None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:43.439057 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15036e7-0bfb-4788-9dac-456b4c946bcd None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:43.439330 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15036e7-0bfb-4788-9dac-456b4c946bcd None admin] PATH_INFO: `/v3/services` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:43.440010 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15036e7-0bfb-4788-9dac-456b4c946bcd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:43.441919 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15036e7-0bfb-4788-9dac-456b4c946bcd None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:43.444271 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d96e48-67ec-4d42-b325-dc6f2305ca00 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:43.444563 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d96e48-67ec-4d42-b325-dc6f2305ca00 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:43.444843 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d96e48-67ec-4d42-b325-dc6f2305ca00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:43.454956 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 480/997] 10.222.0.46 () {56 vars in 1087 bytes} [Thu Jun 23 15:48:43 2022] GET /identity/v3/services => generated 1727 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 15:48:43.463420 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-c0d96e48-67ec-4d42-b325-dc6f2305ca00 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:48:43.464658 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-c0d96e48-67ec-4d42-b325-dc6f2305ca00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:48:43.550873 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 518/998] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jun 23 15:48:43 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 23 15:48:43.571333 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15036e7-0bfb-4788-9dac-456b4c946bcd None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:43.680394 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15036e7-0bfb-4788-9dac-456b4c946bcd None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:43.704065 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:43.716701 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9b324b-cbff-4530-a8e4-56c26246e1e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:43.717530 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9b324b-cbff-4530-a8e4-56c26246e1e4 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:43.717818 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9b324b-cbff-4530-a8e4-56c26246e1e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:43.718100 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9b324b-cbff-4530-a8e4-56c26246e1e4 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:43.718944 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9b324b-cbff-4530-a8e4-56c26246e1e4 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:43.720326 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9b324b-cbff-4530-a8e4-56c26246e1e4 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:43.732573 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-bc9b324b-cbff-4530-a8e4-56c26246e1e4 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 23 15:48:43.732573 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:48:43.732573 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 15:48:43.732573 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 15:48:43.732573 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 15:48:43.732573 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 15:48:43.732573 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 15:48:43.732573 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 15:48:43.732573 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 23 15:48:43.732573 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Jun 23 15:48:43.732573 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 15:48:43.732573 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Jun 23 15:48:43.732573 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:48:43.732573 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:48:43.732573 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:48:43.732573 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:48:43.732573 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:48:43.733544 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:48:43.733544 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:48:43.733544 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:48:43.733544 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:48:43.733544 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:48:43.733544 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:48:43.733544 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 15:48:43.733544 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 15:48:43.733544 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 15:48:43.733544 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 15:48:43.733544 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:48:43.733544 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:48:43.733544 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:48:43.733544 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:48:43.733544 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:48:43.733544 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:48:43.734421 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:48:43.734421 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Jun 23 15:48:43.734421 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:48:43.734421 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:48:43.734421 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:48:43.734421 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:48:43.734421 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:48:43.734421 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:48:43.734421 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:48:43.734421 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:48:43.734421 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 15:48:43.734421 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 15:48:43.734421 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 23 15:48:43.734421 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Jun 23 15:48:43.738775 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 481/999] 10.222.0.46 () {56 vars in 1126 bytes} [Thu Jun 23 15:48:43 2022] GET /identity/v3/services/object-store => generated 93 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:48:43.740757 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d96e48-67ec-4d42-b325-dc6f2305ca00 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:43.849830 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d96e48-67ec-4d42-b325-dc6f2305ca00 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:43.860489 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:43.885652 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd779c9-8c6b-46f1-add0-c267269c012f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:43.886311 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd779c9-8c6b-46f1-add0-c267269c012f None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:43.886590 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd779c9-8c6b-46f1-add0-c267269c012f None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:43.886889 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd779c9-8c6b-46f1-add0-c267269c012f None admin] PATH_INFO: `/v3/services` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:43.887592 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd779c9-8c6b-46f1-add0-c267269c012f None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:48:43.887897 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd779c9-8c6b-46f1-add0-c267269c012f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:43.889266 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd779c9-8c6b-46f1-add0-c267269c012f None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:43.898609 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 519/1000] 10.222.0.46 () {56 vars in 1122 bytes} [Thu Jun 23 15:48:43 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 15:48:43.900464 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9b324b-cbff-4530-a8e4-56c26246e1e4 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:44.004755 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9b324b-cbff-4530-a8e4-56c26246e1e4 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:44.039331 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b1df56-5bb1-476f-87ae-ce0653b8d5e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:44.040008 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b1df56-5bb1-476f-87ae-ce0653b8d5e9 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:44.040294 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b1df56-5bb1-476f-87ae-ce0653b8d5e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:44.040585 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b1df56-5bb1-476f-87ae-ce0653b8d5e9 None admin] PATH_INFO: `/v3/services` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:44.041279 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b1df56-5bb1-476f-87ae-ce0653b8d5e9 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:48:44.041587 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b1df56-5bb1-476f-87ae-ce0653b8d5e9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:44.042978 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b1df56-5bb1-476f-87ae-ce0653b8d5e9 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:44.048012 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 482/1001] 10.222.0.46 () {56 vars in 1122 bytes} [Thu Jun 23 15:48:43 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:48:44.054145 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd779c9-8c6b-46f1-add0-c267269c012f None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:44.161920 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd779c9-8c6b-46f1-add0-c267269c012f None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:44.198776 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e64769c-42c6-4163-b124-e020ad4363e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:44.199449 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e64769c-42c6-4163-b124-e020ad4363e7 None admin] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:44.199730 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e64769c-42c6-4163-b124-e020ad4363e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:44.200006 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e64769c-42c6-4163-b124-e020ad4363e7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:44.200765 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e64769c-42c6-4163-b124-e020ad4363e7 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:44.202142 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e64769c-42c6-4163-b124-e020ad4363e7 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:44.218338 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 520/1002] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Jun 23 15:48:44 2022] POST /identity/v3/endpoints => generated 325 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 15:48:45.105511 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92546dd7-f3b8-4294-afc1-4eecd3ab1f9a None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:45.106057 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92546dd7-f3b8-4294-afc1-4eecd3ab1f9a None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:45.106057 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92546dd7-f3b8-4294-afc1-4eecd3ab1f9a None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:45.107484 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 483/1003] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jun 23 15:48:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 15:48:45.112591 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b025c87f-437f-4278-be43-dc95eb390e1e None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:45.113018 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b025c87f-437f-4278-be43-dc95eb390e1e None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:45.113232 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b025c87f-437f-4278-be43-dc95eb390e1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:45.130238 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-b025c87f-437f-4278-be43-dc95eb390e1e None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:48:45.131425 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-b025c87f-437f-4278-be43-dc95eb390e1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:48:45.158621 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:45.208105 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 521/1004] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jun 23 15:48:45 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 15:48:45.230055 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92546dd7-f3b8-4294-afc1-4eecd3ab1f9a None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:45.285385 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:45.343162 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92546dd7-f3b8-4294-afc1-4eecd3ab1f9a None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:45.377968 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80db4a00-827e-43e7-92df-259fe670e16e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:45.379275 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80db4a00-827e-43e7-92df-259fe670e16e None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:45.379557 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80db4a00-827e-43e7-92df-259fe670e16e None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:45.379831 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80db4a00-827e-43e7-92df-259fe670e16e None admin] PATH_INFO: `/v3/services/image` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:45.380482 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80db4a00-827e-43e7-92df-259fe670e16e None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:45.381847 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80db4a00-827e-43e7-92df-259fe670e16e None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:45.393446 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-80db4a00-827e-43e7-92df-259fe670e16e None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 23 15:48:45.393446 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:48:45.393446 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 15:48:45.393446 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 15:48:45.393446 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 15:48:45.393446 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 15:48:45.393446 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 15:48:45.393446 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 15:48:45.393446 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 23 15:48:45.393446 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Jun 23 15:48:45.393446 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 15:48:45.393446 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Jun 23 15:48:45.393446 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:48:45.393446 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:48:45.393446 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:48:45.393446 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:48:45.393446 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:48:45.394469 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:48:45.394469 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:48:45.394469 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:48:45.394469 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:48:45.394469 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:48:45.394469 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:48:45.394469 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 15:48:45.394469 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 15:48:45.394469 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 15:48:45.394469 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 15:48:45.394469 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:48:45.394469 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:48:45.394469 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:48:45.394469 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:48:45.394469 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:48:45.394469 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:48:45.395668 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:48:45.395668 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Jun 23 15:48:45.395668 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:48:45.395668 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:48:45.395668 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:48:45.395668 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:48:45.395668 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:48:45.395668 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:48:45.395668 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:48:45.395668 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:48:45.395668 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 15:48:45.395668 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 15:48:45.395668 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 23 15:48:45.395668 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Jun 23 15:48:45.396837 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 484/1005] 10.222.0.46 () {56 vars in 1105 bytes} [Thu Jun 23 15:48:45 2022] GET /identity/v3/services/image => generated 86 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:48:45.400768 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b025c87f-437f-4278-be43-dc95eb390e1e None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:45.500171 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b025c87f-437f-4278-be43-dc95eb390e1e None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:45.518351 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83db3ef3-1dbb-407e-9669-6906bad131a8 None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:45.518645 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83db3ef3-1dbb-407e-9669-6906bad131a8 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:45.518971 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83db3ef3-1dbb-407e-9669-6906bad131a8 None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:45.520571 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 485/1006] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jun 23 15:48:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 15:48:45.525455 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d29dc50-12f5-43fa-929a-71386958f7a0 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:45.525754 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d29dc50-12f5-43fa-929a-71386958f7a0 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:45.526032 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d29dc50-12f5-43fa-929a-71386958f7a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:45.536574 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eaa3d34-85d3-4897-8aa3-7a3afd9a2b95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:45.537258 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaa3d34-85d3-4897-8aa3-7a3afd9a2b95 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:45.537537 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaa3d34-85d3-4897-8aa3-7a3afd9a2b95 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:45.537810 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaa3d34-85d3-4897-8aa3-7a3afd9a2b95 None admin] PATH_INFO: `/v3/services` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:45.539070 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eaa3d34-85d3-4897-8aa3-7a3afd9a2b95 None admin] RBAC: Adding query filter params (name=image) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:48:45.539382 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eaa3d34-85d3-4897-8aa3-7a3afd9a2b95 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:45.540917 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eaa3d34-85d3-4897-8aa3-7a3afd9a2b95 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:45.543553 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-7d29dc50-12f5-43fa-929a-71386958f7a0 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:48:45.544764 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-7d29dc50-12f5-43fa-929a-71386958f7a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:48:45.546211 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 522/1007] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Jun 23 15:48:45 2022] GET /identity/v3/services?name=image => generated 122 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 15:48:45.554096 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eaa3d34-85d3-4897-8aa3-7a3afd9a2b95 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:45.586592 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:45.621033 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 486/1008] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jun 23 15:48:45 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 23 15:48:45.644509 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d29dc50-12f5-43fa-929a-71386958f7a0 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:45.668718 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eaa3d34-85d3-4897-8aa3-7a3afd9a2b95 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:45.691813 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:45.706970 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4e0999-331e-4b2c-bb5c-396e59ed8b5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:45.707650 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4e0999-331e-4b2c-bb5c-396e59ed8b5d None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:45.707940 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4e0999-331e-4b2c-bb5c-396e59ed8b5d None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:45.708223 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4e0999-331e-4b2c-bb5c-396e59ed8b5d None admin] PATH_INFO: `/v3/services` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:45.708931 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4e0999-331e-4b2c-bb5c-396e59ed8b5d None admin] RBAC: Adding query filter params (type=image) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:48:45.709243 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4e0999-331e-4b2c-bb5c-396e59ed8b5d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:45.710623 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4e0999-331e-4b2c-bb5c-396e59ed8b5d None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:45.715968 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 523/1009] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Jun 23 15:48:45 2022] GET /identity/v3/services?type=image => generated 122 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:48:45.762622 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d29dc50-12f5-43fa-929a-71386958f7a0 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:45.801884 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca35b0c8-0c49-447b-b794-68f5a9ffae7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:45.802738 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca35b0c8-0c49-447b-b794-68f5a9ffae7d None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:45.803023 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca35b0c8-0c49-447b-b794-68f5a9ffae7d None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:45.803298 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca35b0c8-0c49-447b-b794-68f5a9ffae7d None admin] PATH_INFO: `/v3/services/compute` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:45.803975 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca35b0c8-0c49-447b-b794-68f5a9ffae7d None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:45.805379 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca35b0c8-0c49-447b-b794-68f5a9ffae7d None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:45.817863 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-ca35b0c8-0c49-447b-b794-68f5a9ffae7d None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 23 15:48:45.817863 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:48:45.817863 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 15:48:45.817863 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 15:48:45.817863 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 15:48:45.817863 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 15:48:45.817863 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 15:48:45.817863 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 15:48:45.817863 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 23 15:48:45.817863 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Jun 23 15:48:45.817863 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 15:48:45.817863 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Jun 23 15:48:45.817863 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:48:45.817863 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:48:45.817863 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:48:45.817863 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:48:45.817863 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:48:45.819004 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:48:45.819004 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:48:45.819004 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:48:45.819004 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:48:45.819004 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:48:45.819004 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:48:45.819004 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 15:48:45.819004 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 15:48:45.819004 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 15:48:45.819004 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 15:48:45.819004 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:48:45.819004 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:48:45.819004 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:48:45.819004 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:48:45.819004 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:48:45.819004 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:48:45.825145 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:48:45.825145 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Jun 23 15:48:45.825145 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:48:45.825145 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:48:45.825145 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:48:45.825145 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:48:45.825145 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:48:45.825145 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:48:45.825145 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:48:45.825145 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:48:45.825145 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 15:48:45.825145 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 15:48:45.825145 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 23 15:48:45.825145 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Jun 23 15:48:45.825145 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 487/1010] 10.222.0.46 () {56 vars in 1111 bytes} [Thu Jun 23 15:48:45 2022] GET /identity/v3/services/compute => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:48:45.827322 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4e0999-331e-4b2c-bb5c-396e59ed8b5d None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:45.927633 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:45.940572 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4e0999-331e-4b2c-bb5c-396e59ed8b5d None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:45.976353 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf6a869-5fb4-4067-b33f-0ef42555c2f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:45.977040 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf6a869-5fb4-4067-b33f-0ef42555c2f4 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:45.977327 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf6a869-5fb4-4067-b33f-0ef42555c2f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:45.977605 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf6a869-5fb4-4067-b33f-0ef42555c2f4 None admin] PATH_INFO: `/v3/services` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:45.978325 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf6a869-5fb4-4067-b33f-0ef42555c2f4 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:48:45.978636 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf6a869-5fb4-4067-b33f-0ef42555c2f4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:45.980048 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf6a869-5fb4-4067-b33f-0ef42555c2f4 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:45.990547 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 524/1011] 10.222.0.46 () {56 vars in 1112 bytes} [Thu Jun 23 15:48:45 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:48:45.991759 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca35b0c8-0c49-447b-b794-68f5a9ffae7d None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:46.097153 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca35b0c8-0c49-447b-b794-68f5a9ffae7d None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:46.108738 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:46.133254 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e549b5b6-6ea0-4c1f-b2e9-68d665589de0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:46.133936 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e549b5b6-6ea0-4c1f-b2e9-68d665589de0 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:46.134224 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e549b5b6-6ea0-4c1f-b2e9-68d665589de0 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:46.134496 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e549b5b6-6ea0-4c1f-b2e9-68d665589de0 None admin] PATH_INFO: `/v3/services` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:46.135256 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e549b5b6-6ea0-4c1f-b2e9-68d665589de0 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:48:46.135572 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e549b5b6-6ea0-4c1f-b2e9-68d665589de0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:46.136953 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e549b5b6-6ea0-4c1f-b2e9-68d665589de0 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:46.147254 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 488/1012] 10.222.0.46 () {56 vars in 1112 bytes} [Thu Jun 23 15:48:45 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 15:48:46.149292 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf6a869-5fb4-4067-b33f-0ef42555c2f4 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:46.257355 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf6a869-5fb4-4067-b33f-0ef42555c2f4 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:46.287650 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391889fd-b77c-413a-b85b-deb7bc4d3519 None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:46.288093 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391889fd-b77c-413a-b85b-deb7bc4d3519 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:46.288508 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391889fd-b77c-413a-b85b-deb7bc4d3519 None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:46.290135 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 489/1013] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jun 23 15:48:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 15:48:46.293288 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fb86d5-598a-4812-8252-856d9f09354e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:46.294083 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fb86d5-598a-4812-8252-856d9f09354e None admin] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:46.294261 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fb86d5-598a-4812-8252-856d9f09354e None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:46.294517 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fb86d5-598a-4812-8252-856d9f09354e None admin] PATH_INFO: `/v3/endpoints` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:46.295303 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fb86d5-598a-4812-8252-856d9f09354e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:46.295420 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2acf820-77b7-48f4-ac83-df476ae191fb None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:46.295580 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2acf820-77b7-48f4-ac83-df476ae191fb None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:46.295860 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2acf820-77b7-48f4-ac83-df476ae191fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:46.296677 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fb86d5-598a-4812-8252-856d9f09354e None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:46.314534 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-e2acf820-77b7-48f4-ac83-df476ae191fb None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:48:46.315765 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-e2acf820-77b7-48f4-ac83-df476ae191fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:48:46.317666 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 525/1014] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Jun 23 15:48:46 2022] POST /identity/v3/endpoints => generated 334 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 15:48:46.397987 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 490/1015] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jun 23 15:48:46 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 23 15:48:46.422105 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fb86d5-598a-4812-8252-856d9f09354e None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:46.435340 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:46.536188 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fb86d5-598a-4812-8252-856d9f09354e None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:46.575435 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10dd12a6-43c4-405d-bc0a-1fd5d5d4d1e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:46.576426 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dd12a6-43c4-405d-bc0a-1fd5d5d4d1e3 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:46.576520 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dd12a6-43c4-405d-bc0a-1fd5d5d4d1e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:46.576786 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dd12a6-43c4-405d-bc0a-1fd5d5d4d1e3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:46.578057 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10dd12a6-43c4-405d-bc0a-1fd5d5d4d1e3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:46.579545 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10dd12a6-43c4-405d-bc0a-1fd5d5d4d1e3 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:46.581531 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 526/1016] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Jun 23 15:48:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:48:46.587889 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2acf820-77b7-48f4-ac83-df476ae191fb None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:46.686272 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:46.689989 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2acf820-77b7-48f4-ac83-df476ae191fb None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:46.725448 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5df42e1-9e53-4dad-a16a-b3c89a9a127d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:46.726114 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5df42e1-9e53-4dad-a16a-b3c89a9a127d None admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:46.726397 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5df42e1-9e53-4dad-a16a-b3c89a9a127d None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:46.726668 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5df42e1-9e53-4dad-a16a-b3c89a9a127d None admin] PATH_INFO: `/v3/projects` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:46.727406 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5df42e1-9e53-4dad-a16a-b3c89a9a127d None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:46.728828 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5df42e1-9e53-4dad-a16a-b3c89a9a127d None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:46.755193 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 491/1017] 10.222.0.46 () {60 vars in 1141 bytes} [Thu Jun 23 15:48:46 2022] POST /identity/v3/projects => generated 311 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 15:48:47.802200 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13162d8e-de81-4080-bcf3-61f851e3a65a None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:47.802881 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13162d8e-de81-4080-bcf3-61f851e3a65a None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:47.802881 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13162d8e-de81-4080-bcf3-61f851e3a65a None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:47.804298 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 527/1018] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jun 23 15:48:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 15:48:47.809461 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cacf56-93ee-41ea-8d3b-e0cb0824ffcf None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:47.809843 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cacf56-93ee-41ea-8d3b-e0cb0824ffcf None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:47.810018 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cacf56-93ee-41ea-8d3b-e0cb0824ffcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:47.826990 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-b7cacf56-93ee-41ea-8d3b-e0cb0824ffcf None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:48:47.828118 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-b7cacf56-93ee-41ea-8d3b-e0cb0824ffcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:48:47.904276 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 492/1019] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jun 23 15:48:47 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 15:48:47.927242 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13162d8e-de81-4080-bcf3-61f851e3a65a None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:48.008557 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:48.034795 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13162d8e-de81-4080-bcf3-61f851e3a65a None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:48.068824 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1975ac83-47ef-4196-8b0f-de3b069a455d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:48.069488 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1975ac83-47ef-4196-8b0f-de3b069a455d None admin] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:48.069766 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1975ac83-47ef-4196-8b0f-de3b069a455d None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:48.070038 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1975ac83-47ef-4196-8b0f-de3b069a455d None admin] PATH_INFO: `/v3/services` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:48.070793 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1975ac83-47ef-4196-8b0f-de3b069a455d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:48.072156 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1975ac83-47ef-4196-8b0f-de3b069a455d None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:48.087402 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 528/1020] 10.222.0.46 () {60 vars in 1141 bytes} [Thu Jun 23 15:48:47 2022] POST /identity/v3/services => generated 242 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 15:48:48.489997 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791670cd-0fd1-4be6-ab17-d4c9bd7783a4 None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:48.490197 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791670cd-0fd1-4be6-ab17-d4c9bd7783a4 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:48.490497 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791670cd-0fd1-4be6-ab17-d4c9bd7783a4 None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:48.491938 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 493/1021] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jun 23 15:48:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 15:48:48.497211 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a3d682-3ee1-4f58-8e11-c49e98c170c8 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:48.497504 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a3d682-3ee1-4f58-8e11-c49e98c170c8 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:48.497784 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a3d682-3ee1-4f58-8e11-c49e98c170c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:48.514937 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-d5a3d682-3ee1-4f58-8e11-c49e98c170c8 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:48:48.516128 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-d5a3d682-3ee1-4f58-8e11-c49e98c170c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:48:48.600187 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 529/1022] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jun 23 15:48:48 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 15:48:48.623838 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791670cd-0fd1-4be6-ab17-d4c9bd7783a4 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:48.707489 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:48.728781 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791670cd-0fd1-4be6-ab17-d4c9bd7783a4 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:48.763225 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9a94f5-008a-4b0e-b6e5-2a96b00a3b48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:48.763965 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9a94f5-008a-4b0e-b6e5-2a96b00a3b48 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:48.764249 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9a94f5-008a-4b0e-b6e5-2a96b00a3b48 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:48.764523 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9a94f5-008a-4b0e-b6e5-2a96b00a3b48 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:48.774908 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9a94f5-008a-4b0e-b6e5-2a96b00a3b48 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 15:48:48.783348 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-2d9a94f5-008a-4b0e-b6e5-2a96b00a3b48 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 23 15:48:48.783348 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:48:48.783348 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:48:48.783348 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:48:48.783348 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:48:48.783348 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:48:48.783348 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:48:48.783348 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:48:48.783348 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:48:48.783348 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:48:48.783348 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:48:48.783348 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:48:48.783348 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 15:48:48.783348 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 15:48:48.783348 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 15:48:48.783348 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 15:48:48.786453 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 15:48:48.786453 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 15:48:48.786453 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 15:48:48.786453 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 15:48:48.786453 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:48:48.786453 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:48:48.786453 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:48:48.786453 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:48:48.786453 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:48:48.786453 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:48:48.786453 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:48:48.786453 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Jun 23 15:48:48.786453 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:48:48.786453 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:48:48.786453 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:48:48.786453 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:48:48.787688 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:48:48.787688 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:48:48.787688 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:48:48.787688 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:48:48.787688 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 15:48:48.787688 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 15:48:48.787688 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 15:48:48.787688 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 15:48:48.787688 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 15:48:48.787688 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 15:48:48.787688 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 23 15:48:48.787688 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Jun 23 15:48:48.789226 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 494/1023] 10.222.0.46 () {56 vars in 1120 bytes} [Thu Jun 23 15:48:48 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 15:48:48.791184 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a3d682-3ee1-4f58-8e11-c49e98c170c8 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:48.854584 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:48.877602 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da09cf4c-ec20-451c-aeff-d2a2c9dcbc01 None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:48.878103 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da09cf4c-ec20-451c-aeff-d2a2c9dcbc01 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:48.878183 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da09cf4c-ec20-451c-aeff-d2a2c9dcbc01 None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:48.879836 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 495/1024] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jun 23 15:48:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 15:48:48.886396 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c221dfda-af30-49f8-96c3-82f6714a4b50 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:48.886396 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c221dfda-af30-49f8-96c3-82f6714a4b50 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:48.886396 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c221dfda-af30-49f8-96c3-82f6714a4b50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:48.900492 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a3d682-3ee1-4f58-8e11-c49e98c170c8 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:48.903941 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-c221dfda-af30-49f8-96c3-82f6714a4b50 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:48:48.905384 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-c221dfda-af30-49f8-96c3-82f6714a4b50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:48:48.934769 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e30d8a-c11c-4ceb-a463-0dff37a22726 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:48.935442 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e30d8a-c11c-4ceb-a463-0dff37a22726 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:48.935729 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e30d8a-c11c-4ceb-a463-0dff37a22726 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:48.936005 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e30d8a-c11c-4ceb-a463-0dff37a22726 None admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:48.936752 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e30d8a-c11c-4ceb-a463-0dff37a22726 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:48:48.937060 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e30d8a-c11c-4ceb-a463-0dff37a22726 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:48.938428 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e30d8a-c11c-4ceb-a463-0dff37a22726 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:48.946367 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 530/1025] 10.222.0.46 () {56 vars in 1115 bytes} [Thu Jun 23 15:48:48 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:48:48.951916 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e30d8a-c11c-4ceb-a463-0dff37a22726 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:48.982357 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 496/1026] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jun 23 15:48:48 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 15:48:49.013168 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c221dfda-af30-49f8-96c3-82f6714a4b50 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:49.072987 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e30d8a-c11c-4ceb-a463-0dff37a22726 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:49.087534 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:49.103353 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:49.116333 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98910bc-5ecc-46e6-9162-d6472c285fd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:49.117084 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98910bc-5ecc-46e6-9162-d6472c285fd7 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:49.117381 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98910bc-5ecc-46e6-9162-d6472c285fd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:49.117639 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98910bc-5ecc-46e6-9162-d6472c285fd7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:49.118551 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c221dfda-af30-49f8-96c3-82f6714a4b50 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:49.128714 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98910bc-5ecc-46e6-9162-d6472c285fd7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:49.130175 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98910bc-5ecc-46e6-9162-d6472c285fd7 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:49.148778 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-a98910bc-5ecc-46e6-9162-d6472c285fd7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 15:48:49.148778 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:48:49.148778 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:48:49.148778 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:48:49.148778 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:48:49.148778 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:48:49.148778 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:48:49.148778 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:48:49.148778 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:48:49.148778 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:48:49.148778 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:48:49.148778 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:48:49.148778 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 15:48:49.148778 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 15:48:49.148778 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 15:48:49.150021 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 15:48:49.150021 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:48:49.150021 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:48:49.150021 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:48:49.150021 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:48:49.150021 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:48:49.150021 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:48:49.150021 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:48:49.150021 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Jun 23 15:48:49.150021 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:48:49.150021 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:48:49.150021 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:48:49.150021 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:48:49.150021 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:48:49.150021 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:48:49.150021 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:48:49.150021 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:48:49.150984 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 15:48:49.150984 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 15:48:49.150984 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 15:48:49.150984 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Jun 23 15:48:49.150984 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 531/1027] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Jun 23 15:48:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 15:48:49.155753 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98910bc-5ecc-46e6-9162-d6472c285fd7 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:49.159124 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8523a515-85e1-477b-8464-21bce5ceb400 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:49.160811 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8523a515-85e1-477b-8464-21bce5ceb400 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:49.160811 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8523a515-85e1-477b-8464-21bce5ceb400 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:49.161087 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8523a515-85e1-477b-8464-21bce5ceb400 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:49.162233 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8523a515-85e1-477b-8464-21bce5ceb400 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:49.163718 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8523a515-85e1-477b-8464-21bce5ceb400 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:49.165539 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 497/1028] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Jun 23 15:48:49 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 15:48:49.170934 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8523a515-85e1-477b-8464-21bce5ceb400 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:49.275183 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98910bc-5ecc-46e6-9162-d6472c285fd7 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:49.285639 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8523a515-85e1-477b-8464-21bce5ceb400 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:49.312002 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27714018-a04a-4d8f-8d74-9721e35fea8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:49.312668 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27714018-a04a-4d8f-8d74-9721e35fea8f None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:49.312959 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27714018-a04a-4d8f-8d74-9721e35fea8f None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:49.313236 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27714018-a04a-4d8f-8d74-9721e35fea8f None admin] PATH_INFO: `/v3/domains` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:49.313960 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27714018-a04a-4d8f-8d74-9721e35fea8f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:48:49.314276 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27714018-a04a-4d8f-8d74-9721e35fea8f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:49.315804 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27714018-a04a-4d8f-8d74-9721e35fea8f None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:49.322179 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b19235c-3006-4ab8-82ed-a944288bed5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:49.322896 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b19235c-3006-4ab8-82ed-a944288bed5f None admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:49.323233 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b19235c-3006-4ab8-82ed-a944288bed5f None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:49.323455 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b19235c-3006-4ab8-82ed-a944288bed5f None admin] PATH_INFO: `/v3/users` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:49.324172 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b19235c-3006-4ab8-82ed-a944288bed5f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:49.325589 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b19235c-3006-4ab8-82ed-a944288bed5f None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:49.328250 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 532/1029] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Jun 23 15:48:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 15:48:49.334895 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27714018-a04a-4d8f-8d74-9721e35fea8f None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:49.337405 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:49.359292 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 498/1030] 10.222.0.46 () {60 vars in 1132 bytes} [Thu Jun 23 15:48:49 2022] POST /identity/v3/users => generated 285 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 15:48:49.384352 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:49.457833 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27714018-a04a-4d8f-8d74-9721e35fea8f None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:49.494329 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c15794-2828-4da3-9664-07c853f3fdc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:49.495181 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c15794-2828-4da3-9664-07c853f3fdc7 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:49.495467 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c15794-2828-4da3-9664-07c853f3fdc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:49.495746 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c15794-2828-4da3-9664-07c853f3fdc7 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:49.504422 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c15794-2828-4da3-9664-07c853f3fdc7 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:49.505893 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c15794-2828-4da3-9664-07c853f3fdc7 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:49.521976 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-f5c15794-2828-4da3-9664-07c853f3fdc7 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 23 15:48:49.521976 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:48:49.521976 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:48:49.521976 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:48:49.521976 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:48:49.521976 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:48:49.521976 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:48:49.521976 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:48:49.521976 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:48:49.521976 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:48:49.521976 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:48:49.521976 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:48:49.521976 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 15:48:49.521976 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 15:48:49.521976 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 15:48:49.521976 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 15:48:49.523057 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:48:49.523057 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:48:49.523057 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 15:48:49.523057 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 15:48:49.523057 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 15:48:49.523057 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 15:48:49.523057 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:48:49.523057 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:48:49.523057 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:48:49.523057 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:48:49.523057 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:48:49.523057 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Jun 23 15:48:49.523057 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:48:49.523057 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:48:49.523057 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:48:49.523057 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:48:49.523057 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:48:49.523057 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:48:49.524285 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:48:49.524285 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:48:49.524285 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 15:48:49.524285 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 15:48:49.524285 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 15:48:49.524285 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 15:48:49.524285 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 15:48:49.524285 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 15:48:49.524285 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 23 15:48:49.524285 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Jun 23 15:48:49.528285 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 533/1031] 10.222.0.46 () {56 vars in 1093 bytes} [Thu Jun 23 15:48:49 2022] GET /identity/v3/users/nova => generated 82 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 15:48:49.529699 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b19235c-3006-4ab8-82ed-a944288bed5f None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:49.637174 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b19235c-3006-4ab8-82ed-a944288bed5f None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:49.673794 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d04d67-6755-4965-a328-b593eef22950 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:49.674854 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d04d67-6755-4965-a328-b593eef22950 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:49.675376 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d04d67-6755-4965-a328-b593eef22950 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:49.675856 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d04d67-6755-4965-a328-b593eef22950 None admin] PATH_INFO: `/v3/users` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:49.676892 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d04d67-6755-4965-a328-b593eef22950 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:48:49.677426 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d04d67-6755-4965-a328-b593eef22950 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:49.679089 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d04d67-6755-4965-a328-b593eef22950 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:49.698247 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 499/1032] 10.222.0.46 () {56 vars in 1133 bytes} [Thu Jun 23 15:48:49 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 15:48:49.703765 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c15794-2828-4da3-9664-07c853f3fdc7 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:49.806614 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:49.814171 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c15794-2828-4da3-9664-07c853f3fdc7 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:49.850400 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadbf35a-b2e3-4955-b2d5-15251fe61a3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:49.851085 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadbf35a-b2e3-4955-b2d5-15251fe61a3a None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:49.851375 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadbf35a-b2e3-4955-b2d5-15251fe61a3a None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:49.851654 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadbf35a-b2e3-4955-b2d5-15251fe61a3a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:49.928657 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadbf35a-b2e3-4955-b2d5-15251fe61a3a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:49.930127 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadbf35a-b2e3-4955-b2d5-15251fe61a3a None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:50.035175 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:50.048627 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 534/1033] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Jun 23 15:48:49 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 23 15:48:50.050999 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d04d67-6755-4965-a328-b593eef22950 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:50.110647 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:50.144177 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0569849c-0a2d-40aa-8c67-d5e7065b120a None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:50.144739 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0569849c-0a2d-40aa-8c67-d5e7065b120a None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:50.145208 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0569849c-0a2d-40aa-8c67-d5e7065b120a None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:50.147106 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 535/1034] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jun 23 15:48:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 15:48:50.153308 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01554cf-e998-4486-aa7a-094fbe5adc13 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:50.153596 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01554cf-e998-4486-aa7a-094fbe5adc13 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:50.153873 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01554cf-e998-4486-aa7a-094fbe5adc13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:50.166224 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d04d67-6755-4965-a328-b593eef22950 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:50.173420 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-c01554cf-e998-4486-aa7a-094fbe5adc13 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:48:50.174660 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-c01554cf-e998-4486-aa7a-094fbe5adc13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:48:50.205707 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b08a23c-66b4-4f43-893b-cfea017fa39c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:50.206548 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b08a23c-66b4-4f43-893b-cfea017fa39c None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:50.206880 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b08a23c-66b4-4f43-893b-cfea017fa39c None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:50.207129 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b08a23c-66b4-4f43-893b-cfea017fa39c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:50.222410 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b08a23c-66b4-4f43-893b-cfea017fa39c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:50.224104 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b08a23c-66b4-4f43-893b-cfea017fa39c None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:50.243182 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-2b08a23c-66b4-4f43-893b-cfea017fa39c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 15:48:50.243182 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:48:50.243182 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:48:50.243182 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:48:50.243182 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:48:50.243182 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:48:50.243182 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:48:50.243182 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:48:50.243182 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:48:50.243182 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:48:50.243182 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:48:50.243182 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:48:50.243182 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 15:48:50.243182 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 15:48:50.243182 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 15:48:50.244332 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 15:48:50.244332 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:48:50.244332 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:48:50.244332 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:48:50.244332 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:48:50.244332 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:48:50.244332 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:48:50.244332 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:48:50.244332 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Jun 23 15:48:50.244332 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:48:50.244332 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:48:50.244332 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:48:50.244332 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:48:50.244332 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:48:50.244332 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:48:50.244332 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:48:50.244332 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:48:50.245477 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 15:48:50.245477 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 15:48:50.245477 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 15:48:50.245477 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Jun 23 15:48:50.249198 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 500/1035] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Jun 23 15:48:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 15:48:50.249936 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b08a23c-66b4-4f43-893b-cfea017fa39c None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:50.257961 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 536/1036] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jun 23 15:48:50 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 23 15:48:50.275684 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01554cf-e998-4486-aa7a-094fbe5adc13 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:50.363746 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b08a23c-66b4-4f43-893b-cfea017fa39c None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:50.371847 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:50.394266 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01554cf-e998-4486-aa7a-094fbe5adc13 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:50.402681 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475e0ce8-1b6b-4987-8829-a287b6c5af84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:50.403399 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475e0ce8-1b6b-4987-8829-a287b6c5af84 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:50.403689 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475e0ce8-1b6b-4987-8829-a287b6c5af84 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:50.403963 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475e0ce8-1b6b-4987-8829-a287b6c5af84 None admin] PATH_INFO: `/v3/domains` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:50.404823 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475e0ce8-1b6b-4987-8829-a287b6c5af84 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:48:50.405132 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475e0ce8-1b6b-4987-8829-a287b6c5af84 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:50.406633 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475e0ce8-1b6b-4987-8829-a287b6c5af84 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:50.420202 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 501/1037] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Jun 23 15:48:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:48:50.426493 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475e0ce8-1b6b-4987-8829-a287b6c5af84 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:50.429319 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:50.445323 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d523114c-979e-44fa-9d58-8ab3de9e5c4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:50.447963 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d523114c-979e-44fa-9d58-8ab3de9e5c4a None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:50.447963 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d523114c-979e-44fa-9d58-8ab3de9e5c4a None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:50.447963 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d523114c-979e-44fa-9d58-8ab3de9e5c4a None admin] PATH_INFO: `/v3/services/image` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:50.449141 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d523114c-979e-44fa-9d58-8ab3de9e5c4a None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:50.452009 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d523114c-979e-44fa-9d58-8ab3de9e5c4a None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:50.474612 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-d523114c-979e-44fa-9d58-8ab3de9e5c4a None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 23 15:48:50.474612 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:48:50.474612 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 15:48:50.474612 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 15:48:50.474612 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 15:48:50.474612 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 15:48:50.474612 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 15:48:50.474612 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 15:48:50.474612 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 23 15:48:50.474612 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Jun 23 15:48:50.474612 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 15:48:50.474612 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Jun 23 15:48:50.474612 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:48:50.474612 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:48:50.474612 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:48:50.474612 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:48:50.474612 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:48:50.475717 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:48:50.475717 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:48:50.475717 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:48:50.475717 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:48:50.475717 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:48:50.475717 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:48:50.475717 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 15:48:50.475717 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 15:48:50.475717 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 15:48:50.475717 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 15:48:50.475717 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:48:50.475717 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:48:50.475717 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:48:50.475717 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:48:50.475717 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:48:50.475717 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:48:50.476592 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:48:50.476592 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Jun 23 15:48:50.476592 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:48:50.476592 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:48:50.476592 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:48:50.476592 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:48:50.476592 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:48:50.476592 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:48:50.476592 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:48:50.476592 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:48:50.476592 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 15:48:50.476592 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 15:48:50.476592 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 23 15:48:50.476592 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Jun 23 15:48:50.476592 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 537/1038] 10.222.0.46 () {56 vars in 1105 bytes} [Thu Jun 23 15:48:50 2022] GET /identity/v3/services/image => generated 86 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 15:48:50.484413 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d523114c-979e-44fa-9d58-8ab3de9e5c4a None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:50.570121 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475e0ce8-1b6b-4987-8829-a287b6c5af84 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:50.613140 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6669ba9-4a05-42f8-a16d-98838bda0ee5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:50.614030 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6669ba9-4a05-42f8-a16d-98838bda0ee5 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:50.614232 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6669ba9-4a05-42f8-a16d-98838bda0ee5 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:50.614509 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6669ba9-4a05-42f8-a16d-98838bda0ee5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:50.615686 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d523114c-979e-44fa-9d58-8ab3de9e5c4a None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:50.626377 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6669ba9-4a05-42f8-a16d-98838bda0ee5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:50.627859 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6669ba9-4a05-42f8-a16d-98838bda0ee5 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:50.647360 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-d6669ba9-4a05-42f8-a16d-98838bda0ee5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 15:48:50.647360 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:48:50.647360 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:48:50.647360 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:48:50.647360 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:48:50.647360 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:48:50.647360 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:48:50.647360 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:48:50.647360 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:48:50.647360 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:48:50.647360 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:48:50.647360 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:48:50.647360 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 15:48:50.647360 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 15:48:50.647360 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 15:48:50.648348 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 15:48:50.648348 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:48:50.648348 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:48:50.648348 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:48:50.648348 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:48:50.648348 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:48:50.648348 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:48:50.648348 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:48:50.648348 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Jun 23 15:48:50.648348 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:48:50.648348 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:48:50.648348 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:48:50.648348 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:48:50.648348 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:48:50.648348 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:48:50.648348 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:48:50.648348 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:48:50.650224 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 15:48:50.650224 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 15:48:50.650224 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 15:48:50.650224 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 15:48:50.650224 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 15:48:50.650224 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 15:48:50.650224 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 15:48:50.650224 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Jun 23 15:48:50.650224 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 502/1039] 10.222.0.46 () {56 vars in 1111 bytes} [Thu Jun 23 15:48:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 15:48:50.654616 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6669ba9-4a05-42f8-a16d-98838bda0ee5 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:50.662118 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220b05ea-264e-4fca-b67d-90f8b64b71f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:50.663103 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220b05ea-264e-4fca-b67d-90f8b64b71f3 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:50.663547 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220b05ea-264e-4fca-b67d-90f8b64b71f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:50.663829 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220b05ea-264e-4fca-b67d-90f8b64b71f3 None admin] PATH_INFO: `/v3/services` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:50.664564 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220b05ea-264e-4fca-b67d-90f8b64b71f3 None admin] RBAC: Adding query filter params (name=image) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:48:50.664875 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220b05ea-264e-4fca-b67d-90f8b64b71f3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:50.666273 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220b05ea-264e-4fca-b67d-90f8b64b71f3 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:50.677542 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 538/1040] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Jun 23 15:48:50 2022] GET /identity/v3/services?name=image => generated 122 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:48:50.679688 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220b05ea-264e-4fca-b67d-90f8b64b71f3 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:50.688939 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:50.737262 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:50.771963 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6669ba9-4a05-42f8-a16d-98838bda0ee5 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:50.788108 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220b05ea-264e-4fca-b67d-90f8b64b71f3 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:50.808780 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7b2e35-e8c8-4086-8709-13cc1efba8d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:50.809688 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7b2e35-e8c8-4086-8709-13cc1efba8d0 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:50.810125 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7b2e35-e8c8-4086-8709-13cc1efba8d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:50.810552 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7b2e35-e8c8-4086-8709-13cc1efba8d0 None admin] PATH_INFO: `/v3/projects` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:50.811517 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7b2e35-e8c8-4086-8709-13cc1efba8d0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:48:50.811971 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7b2e35-e8c8-4086-8709-13cc1efba8d0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:50.813561 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7b2e35-e8c8-4086-8709-13cc1efba8d0 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:50.824131 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3812bd-7beb-48b2-96b6-cda1e3c112c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:50.824796 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3812bd-7beb-48b2-96b6-cda1e3c112c3 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:50.825076 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3812bd-7beb-48b2-96b6-cda1e3c112c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:50.825354 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3812bd-7beb-48b2-96b6-cda1e3c112c3 None admin] PATH_INFO: `/v3/services` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:50.826057 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3812bd-7beb-48b2-96b6-cda1e3c112c3 None admin] RBAC: Adding query filter params (type=image) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:48:50.826378 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3812bd-7beb-48b2-96b6-cda1e3c112c3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:50.827764 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3812bd-7beb-48b2-96b6-cda1e3c112c3 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:50.829141 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 503/1041] 10.222.0.46 () {56 vars in 1148 bytes} [Thu Jun 23 15:48:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 15:48:50.834751 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7b2e35-e8c8-4086-8709-13cc1efba8d0 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:50.837868 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 539/1042] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Jun 23 15:48:50 2022] GET /identity/v3/services?type=image => generated 350 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 15:48:50.844538 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3812bd-7beb-48b2-96b6-cda1e3c112c3 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:50.928332 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:50.944333 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7b2e35-e8c8-4086-8709-13cc1efba8d0 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:50.949952 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3812bd-7beb-48b2-96b6-cda1e3c112c3 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:50.968153 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:50.980518 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffec6802-ec83-4774-b524-a127b5209465 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:50.981190 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffec6802-ec83-4774-b524-a127b5209465 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:50.981478 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffec6802-ec83-4774-b524-a127b5209465 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:50.981759 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffec6802-ec83-4774-b524-a127b5209465 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:50.982600 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffec6802-ec83-4774-b524-a127b5209465 None admin] RBAC: Adding query filter params (role.id=50fafe10bef649b4b87dfab0e16e7915, scope.project.id=28ecdf5b474c44098c722f677d2a0b77, user.id=705115d19bab424d894bea8b1b48a7f0) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:48:50.982946 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffec6802-ec83-4774-b524-a127b5209465 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:50.984393 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffec6802-ec83-4774-b524-a127b5209465 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:50.988493 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b5d276-4cd8-4a36-ae22-b42d2c833202 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:50.989325 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b5d276-4cd8-4a36-ae22-b42d2c833202 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:50.989573 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b5d276-4cd8-4a36-ae22-b42d2c833202 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:50.989780 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b5d276-4cd8-4a36-ae22-b42d2c833202 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:50.990570 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b5d276-4cd8-4a36-ae22-b42d2c833202 None admin] RBAC: Adding query filter params (interface=public, service_id=d67b80b4f471484c841dd1726853d95b, region_id=RegionOne) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:48:50.990911 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b5d276-4cd8-4a36-ae22-b42d2c833202 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:50.991550 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 504/1043] 10.222.0.46 () {56 vars in 1374 bytes} [Thu Jun 23 15:48:50 2022] GET /identity/v3/role_assignments?user.id=705115d19bab424d894bea8b1b48a7f0&scope.project.id=28ecdf5b474c44098c722f677d2a0b77&role.id=50fafe10bef649b4b87dfab0e16e7915 => generated 259 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 15:48:50.995816 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b5d276-4cd8-4a36-ae22-b42d2c833202 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:51.002234 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 540/1044] 10.222.0.46 () {56 vars in 1251 bytes} [Thu Jun 23 15:48:50 2022] GET /identity/v3/endpoints?service_id=d67b80b4f471484c841dd1726853d95b&interface=public®ion_id=RegionOne => generated 194 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 15:48:51.008159 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffec6802-ec83-4774-b524-a127b5209465 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:51.121500 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffec6802-ec83-4774-b524-a127b5209465 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:51.157661 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c65994-c260-451c-869c-6c0859c2770e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:51.158339 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c65994-c260-451c-869c-6c0859c2770e None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:51.158620 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c65994-c260-451c-869c-6c0859c2770e None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:51.158920 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c65994-c260-451c-869c-6c0859c2770e None admin] PATH_INFO: `/v3/services` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:51.160130 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c65994-c260-451c-869c-6c0859c2770e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:51.161490 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c65994-c260-451c-869c-6c0859c2770e None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:51.168382 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 505/1045] 10.222.0.46 () {56 vars in 1087 bytes} [Thu Jun 23 15:48:51 2022] GET /identity/v3/services => generated 1957 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 15:48:51.540868 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e8b8aa-a151-4bd5-a0b2-133c3f48ea92 None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:51.541244 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e8b8aa-a151-4bd5-a0b2-133c3f48ea92 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:51.541447 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e8b8aa-a151-4bd5-a0b2-133c3f48ea92 None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:51.542885 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 541/1046] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jun 23 15:48:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 15:48:51.548029 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db278a2-0b2b-4c5b-baae-7d5543d8e7c4 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:51.548326 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db278a2-0b2b-4c5b-baae-7d5543d8e7c4 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:51.548598 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db278a2-0b2b-4c5b-baae-7d5543d8e7c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:51.569350 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-9db278a2-0b2b-4c5b-baae-7d5543d8e7c4 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:48:51.570601 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-9db278a2-0b2b-4c5b-baae-7d5543d8e7c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:48:51.582755 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:51.642738 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 506/1047] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jun 23 15:48:51 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 15:48:51.666933 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e8b8aa-a151-4bd5-a0b2-133c3f48ea92 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:51.772160 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e8b8aa-a151-4bd5-a0b2-133c3f48ea92 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:51.806912 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e1fad5-5eac-4803-88ba-35523fbfb6c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:51.807633 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e1fad5-5eac-4803-88ba-35523fbfb6c3 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:51.807926 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e1fad5-5eac-4803-88ba-35523fbfb6c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:51.808204 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e1fad5-5eac-4803-88ba-35523fbfb6c3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:51.819298 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e1fad5-5eac-4803-88ba-35523fbfb6c3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 15:48:51.827029 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-41e1fad5-5eac-4803-88ba-35523fbfb6c3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 15:48:51.827029 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:48:51.827029 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:48:51.827029 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:48:51.827029 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:48:51.827029 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:48:51.827029 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:48:51.827029 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:48:51.827029 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:48:51.827029 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:48:51.827029 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:48:51.827029 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:48:51.827029 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 15:48:51.827029 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 15:48:51.827029 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 15:48:51.827029 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 15:48:51.827932 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 15:48:51.827932 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 15:48:51.827932 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 15:48:51.827932 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 15:48:51.827932 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:48:51.827932 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:48:51.827932 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:48:51.827932 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:48:51.827932 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:48:51.827932 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:48:51.827932 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:48:51.827932 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Jun 23 15:48:51.827932 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:48:51.827932 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:48:51.827932 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:48:51.827932 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:48:51.828777 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:48:51.828777 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:48:51.828777 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:48:51.828777 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:48:51.828777 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 15:48:51.828777 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 15:48:51.828777 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 15:48:51.828777 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 15:48:51.828777 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 15:48:51.828777 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 15:48:51.828777 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 15:48:51.828777 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Jun 23 15:48:51.836429 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 542/1048] 10.222.0.46 () {56 vars in 1096 bytes} [Thu Jun 23 15:48:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:48:51.836572 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db278a2-0b2b-4c5b-baae-7d5543d8e7c4 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:51.934763 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db278a2-0b2b-4c5b-baae-7d5543d8e7c4 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:51.968084 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac5215d-0f96-4692-8b5c-d4b5e0372535 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:51.968826 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac5215d-0f96-4692-8b5c-d4b5e0372535 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:51.969117 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac5215d-0f96-4692-8b5c-d4b5e0372535 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:51.969395 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac5215d-0f96-4692-8b5c-d4b5e0372535 None admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:51.970147 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac5215d-0f96-4692-8b5c-d4b5e0372535 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:48:51.970459 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac5215d-0f96-4692-8b5c-d4b5e0372535 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:51.972518 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac5215d-0f96-4692-8b5c-d4b5e0372535 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:51.985213 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 507/1049] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Jun 23 15:48:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:48:51.986407 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e1fad5-5eac-4803-88ba-35523fbfb6c3 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:52.031590 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:52.092055 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e1fad5-5eac-4803-88ba-35523fbfb6c3 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:52.125623 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c067d165-db5e-42b7-aa5b-2fad1fc01d0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:52.126499 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c067d165-db5e-42b7-aa5b-2fad1fc01d0b None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:52.126801 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c067d165-db5e-42b7-aa5b-2fad1fc01d0b None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:52.127078 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c067d165-db5e-42b7-aa5b-2fad1fc01d0b None admin] PATH_INFO: `/v3/users/d86946037a1d41ea943b1b598457b267` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:52.140330 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c067d165-db5e-42b7-aa5b-2fad1fc01d0b None admin] RBAC: Authorizing `identity:get_user(user_id=d86946037a1d41ea943b1b598457b267)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:52.141765 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c067d165-db5e-42b7-aa5b-2fad1fc01d0b None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:52.148573 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 543/1050] 10.222.0.46 () {56 vars in 1177 bytes} [Thu Jun 23 15:48:51 2022] GET /identity/v3/users/d86946037a1d41ea943b1b598457b267 => generated 285 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 15:48:52.150057 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac5215d-0f96-4692-8b5c-d4b5e0372535 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:52.161543 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:52.255643 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac5215d-0f96-4692-8b5c-d4b5e0372535 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:52.292396 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890e6d6f-df57-467e-bb35-5ffa81819139 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:52.293076 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890e6d6f-df57-467e-bb35-5ffa81819139 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:52.293360 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890e6d6f-df57-467e-bb35-5ffa81819139 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:52.293637 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890e6d6f-df57-467e-bb35-5ffa81819139 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:52.370369 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890e6d6f-df57-467e-bb35-5ffa81819139 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:52.371839 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890e6d6f-df57-467e-bb35-5ffa81819139 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:52.380163 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:52.478706 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 508/1051] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Jun 23 15:48:52 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 23 15:48:52.484525 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c067d165-db5e-42b7-aa5b-2fad1fc01d0b None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:52.587699 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:52.590476 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c067d165-db5e-42b7-aa5b-2fad1fc01d0b None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:52.625834 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd9019d-03eb-4fc1-bd5f-3588616a38cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:52.626625 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd9019d-03eb-4fc1-bd5f-3588616a38cb None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:52.626937 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd9019d-03eb-4fc1-bd5f-3588616a38cb None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:52.627247 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd9019d-03eb-4fc1-bd5f-3588616a38cb None admin] PATH_INFO: `/v3/projects/68a2c61ef29c405e9509ad7cc3121def` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:52.628956 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd9019d-03eb-4fc1-bd5f-3588616a38cb None admin] RBAC: Authorizing `identity:get_project(project_id=68a2c61ef29c405e9509ad7cc3121def)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:52.630402 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd9019d-03eb-4fc1-bd5f-3588616a38cb None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:52.637116 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 544/1052] 10.222.0.46 () {56 vars in 1186 bytes} [Thu Jun 23 15:48:52 2022] GET /identity/v3/projects/68a2c61ef29c405e9509ad7cc3121def => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:48:52.638717 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890e6d6f-df57-467e-bb35-5ffa81819139 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:52.749810 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890e6d6f-df57-467e-bb35-5ffa81819139 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:52.758681 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:52.791775 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa55e1c7-67ec-4e2c-9909-ddbe2a93483b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:52.792444 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa55e1c7-67ec-4e2c-9909-ddbe2a93483b None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:52.792729 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa55e1c7-67ec-4e2c-9909-ddbe2a93483b None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:52.793011 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa55e1c7-67ec-4e2c-9909-ddbe2a93483b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:52.793861 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa55e1c7-67ec-4e2c-9909-ddbe2a93483b None admin] RBAC: Adding query filter params (role.id=812d3c567c52469cb2b35709fbdc857c, scope.project.id=68a2c61ef29c405e9509ad7cc3121def, user.id=d86946037a1d41ea943b1b598457b267) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:48:52.794179 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa55e1c7-67ec-4e2c-9909-ddbe2a93483b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:52.795652 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa55e1c7-67ec-4e2c-9909-ddbe2a93483b None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:52.810808 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 509/1053] 10.222.0.46 () {56 vars in 1374 bytes} [Thu Jun 23 15:48:52 2022] GET /identity/v3/role_assignments?user.id=d86946037a1d41ea943b1b598457b267&scope.project.id=68a2c61ef29c405e9509ad7cc3121def&role.id=812d3c567c52469cb2b35709fbdc857c => generated 259 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:48:53.026739 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccde440-95db-44af-88ec-0743c5355d45 None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:53.027240 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccde440-95db-44af-88ec-0743c5355d45 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:53.027240 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccde440-95db-44af-88ec-0743c5355d45 None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:53.028791 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 545/1054] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jun 23 15:48:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 15:48:53.033964 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2266cf81-1c97-4ee1-8465-a816b9e58600 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:53.034296 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2266cf81-1c97-4ee1-8465-a816b9e58600 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:53.034585 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2266cf81-1c97-4ee1-8465-a816b9e58600 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:53.053190 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-2266cf81-1c97-4ee1-8465-a816b9e58600 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:48:53.054412 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-2266cf81-1c97-4ee1-8465-a816b9e58600 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:48:53.127455 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 510/1055] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jun 23 15:48:53 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 15:48:53.150808 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccde440-95db-44af-88ec-0743c5355d45 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:53.264130 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccde440-95db-44af-88ec-0743c5355d45 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:53.285538 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46226972-38aa-493e-9dbf-645ca3560e6c None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:53.286060 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46226972-38aa-493e-9dbf-645ca3560e6c None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:53.286060 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46226972-38aa-493e-9dbf-645ca3560e6c None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:53.287579 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 511/1056] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jun 23 15:48:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 15:48:53.292468 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85656885-f76b-4624-bb90-7b1e730b7853 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:53.292762 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85656885-f76b-4624-bb90-7b1e730b7853 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:53.293040 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85656885-f76b-4624-bb90-7b1e730b7853 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:53.302734 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0e02ac-bfe6-462c-96f4-8d680ee99651 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:53.303563 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0e02ac-bfe6-462c-96f4-8d680ee99651 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:53.303848 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0e02ac-bfe6-462c-96f4-8d680ee99651 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:53.304123 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0e02ac-bfe6-462c-96f4-8d680ee99651 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:53.310385 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-85656885-f76b-4624-bb90-7b1e730b7853 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:48:53.311626 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-85656885-f76b-4624-bb90-7b1e730b7853 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:48:53.316147 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0e02ac-bfe6-462c-96f4-8d680ee99651 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 15:48:53.324428 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-eb0e02ac-bfe6-462c-96f4-8d680ee99651 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 23 15:48:53.324428 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:48:53.324428 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:48:53.324428 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:48:53.324428 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:48:53.324428 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:48:53.324428 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:48:53.324428 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:48:53.324428 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:48:53.324428 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:48:53.324428 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:48:53.324428 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:48:53.324428 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 15:48:53.324428 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 15:48:53.324428 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 15:48:53.324428 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 15:48:53.325398 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 15:48:53.325398 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 15:48:53.325398 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 15:48:53.325398 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 15:48:53.325398 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:48:53.325398 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:48:53.325398 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:48:53.325398 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:48:53.325398 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:48:53.325398 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:48:53.325398 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:48:53.325398 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Jun 23 15:48:53.325398 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:48:53.325398 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:48:53.325398 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:48:53.325398 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:48:53.326242 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:48:53.326242 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:48:53.326242 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:48:53.326242 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:48:53.326242 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 15:48:53.326242 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 15:48:53.326242 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 15:48:53.326242 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 15:48:53.326242 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 15:48:53.326242 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 15:48:53.326242 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 23 15:48:53.326242 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Jun 23 15:48:53.326242 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 546/1057] 10.222.0.46 () {56 vars in 1120 bytes} [Thu Jun 23 15:48:53 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:48:53.331778 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0e02ac-bfe6-462c-96f4-8d680ee99651 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:53.361462 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:53.389688 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 512/1058] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jun 23 15:48:53 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 23 15:48:53.413858 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85656885-f76b-4624-bb90-7b1e730b7853 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:53.445980 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:53.454979 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0e02ac-bfe6-462c-96f4-8d680ee99651 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:53.492201 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e325bf0b-bde8-4829-b9a9-593b582d7915 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:53.492894 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e325bf0b-bde8-4829-b9a9-593b582d7915 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:53.493183 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e325bf0b-bde8-4829-b9a9-593b582d7915 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:53.493466 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e325bf0b-bde8-4829-b9a9-593b582d7915 None admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:53.494234 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e325bf0b-bde8-4829-b9a9-593b582d7915 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:48:53.494545 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e325bf0b-bde8-4829-b9a9-593b582d7915 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:53.496035 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e325bf0b-bde8-4829-b9a9-593b582d7915 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:53.505455 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 547/1059] 10.222.0.46 () {56 vars in 1115 bytes} [Thu Jun 23 15:48:53 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 15:48:53.511240 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e325bf0b-bde8-4829-b9a9-593b582d7915 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:53.545195 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85656885-f76b-4624-bb90-7b1e730b7853 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:53.589512 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3255c98-34b9-45c0-8673-a99e6b3ce663 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:53.590381 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3255c98-34b9-45c0-8673-a99e6b3ce663 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:53.590669 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3255c98-34b9-45c0-8673-a99e6b3ce663 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:53.590979 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3255c98-34b9-45c0-8673-a99e6b3ce663 None admin] PATH_INFO: `/v3/services/image` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:53.591641 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3255c98-34b9-45c0-8673-a99e6b3ce663 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:53.593161 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3255c98-34b9-45c0-8673-a99e6b3ce663 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:53.605572 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-b3255c98-34b9-45c0-8673-a99e6b3ce663 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 23 15:48:53.605572 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:48:53.605572 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 15:48:53.605572 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 15:48:53.605572 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 15:48:53.605572 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 15:48:53.605572 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 15:48:53.605572 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 15:48:53.605572 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 23 15:48:53.605572 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Jun 23 15:48:53.605572 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 15:48:53.605572 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Jun 23 15:48:53.605572 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:48:53.605572 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:48:53.605572 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:48:53.605572 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:48:53.605572 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:48:53.607150 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:48:53.607150 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:48:53.607150 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:48:53.607150 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:48:53.607150 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:48:53.607150 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:48:53.607150 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 15:48:53.607150 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 15:48:53.607150 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 15:48:53.607150 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 15:48:53.607150 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:48:53.607150 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:48:53.607150 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:48:53.607150 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:48:53.607150 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:48:53.607150 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:48:53.608183 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:48:53.608183 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Jun 23 15:48:53.608183 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:48:53.608183 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:48:53.608183 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:48:53.608183 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:48:53.608183 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:48:53.608183 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:48:53.608183 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:48:53.608183 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:48:53.608183 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 15:48:53.608183 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 15:48:53.608183 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 23 15:48:53.608183 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Jun 23 15:48:53.608183 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 513/1060] 10.222.0.46 () {56 vars in 1105 bytes} [Thu Jun 23 15:48:53 2022] GET /identity/v3/services/image => generated 86 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:48:53.614829 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3255c98-34b9-45c0-8673-a99e6b3ce663 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:53.629803 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:53.641554 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e325bf0b-bde8-4829-b9a9-593b582d7915 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:53.678658 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2daf1a9b-e53f-4fdf-808b-08a9bd2b9f4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:53.679539 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2daf1a9b-e53f-4fdf-808b-08a9bd2b9f4f None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:53.680062 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2daf1a9b-e53f-4fdf-808b-08a9bd2b9f4f None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:53.680192 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2daf1a9b-e53f-4fdf-808b-08a9bd2b9f4f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:53.691695 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2daf1a9b-e53f-4fdf-808b-08a9bd2b9f4f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:53.693154 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2daf1a9b-e53f-4fdf-808b-08a9bd2b9f4f None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:53.710998 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-2daf1a9b-e53f-4fdf-808b-08a9bd2b9f4f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 15:48:53.710998 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:48:53.710998 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:48:53.710998 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:48:53.710998 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:48:53.710998 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:48:53.710998 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:48:53.710998 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:48:53.710998 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:48:53.710998 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:48:53.710998 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:48:53.710998 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:48:53.710998 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 15:48:53.710998 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 15:48:53.710998 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 15:48:53.711930 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 15:48:53.711930 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:48:53.711930 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:48:53.711930 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:48:53.711930 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:48:53.711930 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:48:53.711930 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:48:53.711930 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:48:53.711930 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Jun 23 15:48:53.711930 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:48:53.711930 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:48:53.711930 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:48:53.711930 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:48:53.711930 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:48:53.711930 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:48:53.711930 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:48:53.711930 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:48:53.712833 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 15:48:53.712833 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 15:48:53.712833 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 15:48:53.712833 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Jun 23 15:48:53.712833 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 548/1061] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Jun 23 15:48:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 15:48:53.718116 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2daf1a9b-e53f-4fdf-808b-08a9bd2b9f4f None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:53.724609 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:53.733387 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3255c98-34b9-45c0-8673-a99e6b3ce663 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:53.773234 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac0e067-ea8d-46e5-a24e-f875148b3eac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:53.774148 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac0e067-ea8d-46e5-a24e-f875148b3eac None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:53.774441 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac0e067-ea8d-46e5-a24e-f875148b3eac None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:53.774742 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac0e067-ea8d-46e5-a24e-f875148b3eac None admin] PATH_INFO: `/v3/services` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:53.775462 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac0e067-ea8d-46e5-a24e-f875148b3eac None admin] RBAC: Adding query filter params (name=image) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:48:53.775770 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac0e067-ea8d-46e5-a24e-f875148b3eac None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:53.777138 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac0e067-ea8d-46e5-a24e-f875148b3eac None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:53.787557 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 514/1062] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Jun 23 15:48:53 2022] GET /identity/v3/services?name=image => generated 122 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 15:48:53.789782 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac0e067-ea8d-46e5-a24e-f875148b3eac None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:53.848592 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2daf1a9b-e53f-4fdf-808b-08a9bd2b9f4f None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:53.889302 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc54ed6-5d5f-483a-a621-8b08dfde372e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:53.889988 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc54ed6-5d5f-483a-a621-8b08dfde372e None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:53.890270 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc54ed6-5d5f-483a-a621-8b08dfde372e None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:53.890548 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc54ed6-5d5f-483a-a621-8b08dfde372e None admin] PATH_INFO: `/v3/domains` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:53.891341 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc54ed6-5d5f-483a-a621-8b08dfde372e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:48:53.891650 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc54ed6-5d5f-483a-a621-8b08dfde372e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:53.893086 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc54ed6-5d5f-483a-a621-8b08dfde372e None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:53.910298 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac0e067-ea8d-46e5-a24e-f875148b3eac None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:53.915418 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 549/1063] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Jun 23 15:48:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 15:48:53.916530 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc54ed6-5d5f-483a-a621-8b08dfde372e None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:53.919713 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:53.955261 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdc9d83-3dcf-4638-8419-aaa7d505875a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:53.955965 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdc9d83-3dcf-4638-8419-aaa7d505875a None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:53.956831 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdc9d83-3dcf-4638-8419-aaa7d505875a None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:53.957112 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdc9d83-3dcf-4638-8419-aaa7d505875a None admin] PATH_INFO: `/v3/services` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:53.957842 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdc9d83-3dcf-4638-8419-aaa7d505875a None admin] RBAC: Adding query filter params (type=image) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:48:53.958151 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdc9d83-3dcf-4638-8419-aaa7d505875a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:53.959611 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdc9d83-3dcf-4638-8419-aaa7d505875a None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:53.970280 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 515/1064] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Jun 23 15:48:53 2022] GET /identity/v3/services?type=image => generated 350 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 15:48:53.971535 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdc9d83-3dcf-4638-8419-aaa7d505875a None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:54.000451 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:54.036575 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc54ed6-5d5f-483a-a621-8b08dfde372e None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:54.073199 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638d846d-5e54-4968-a564-293bdf404e46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:54.074012 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638d846d-5e54-4968-a564-293bdf404e46 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:54.074298 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638d846d-5e54-4968-a564-293bdf404e46 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:54.074576 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638d846d-5e54-4968-a564-293bdf404e46 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:54.083298 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638d846d-5e54-4968-a564-293bdf404e46 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:54.084765 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638d846d-5e54-4968-a564-293bdf404e46 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:54.085681 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdc9d83-3dcf-4638-8419-aaa7d505875a None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:54.100380 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-638d846d-5e54-4968-a564-293bdf404e46 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 23 15:48:54.100380 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:48:54.100380 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:48:54.100380 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:48:54.100380 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:48:54.100380 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:48:54.100380 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:48:54.100380 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:48:54.100380 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:48:54.100380 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:48:54.100380 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:48:54.100380 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:48:54.100380 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 15:48:54.100380 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 15:48:54.100380 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 15:48:54.100380 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 15:48:54.101571 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:48:54.101571 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:48:54.101571 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 15:48:54.101571 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 15:48:54.101571 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 15:48:54.101571 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 15:48:54.101571 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:48:54.101571 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:48:54.101571 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:48:54.101571 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:48:54.101571 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:48:54.101571 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Jun 23 15:48:54.101571 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:48:54.101571 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:48:54.101571 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:48:54.101571 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:48:54.101571 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:48:54.101571 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:48:54.102536 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:48:54.102536 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:48:54.102536 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 15:48:54.102536 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 15:48:54.102536 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 15:48:54.102536 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 15:48:54.102536 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 15:48:54.102536 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 15:48:54.102536 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 23 15:48:54.102536 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Jun 23 15:48:54.102536 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 550/1065] 10.222.0.46 () {56 vars in 1093 bytes} [Thu Jun 23 15:48:53 2022] GET /identity/v3/users/nova => generated 82 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:48:54.109472 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638d846d-5e54-4968-a564-293bdf404e46 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:54.129725 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8cc521-973c-4809-b32d-17932df68f46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:54.130401 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8cc521-973c-4809-b32d-17932df68f46 None admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:54.130700 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8cc521-973c-4809-b32d-17932df68f46 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:54.131001 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8cc521-973c-4809-b32d-17932df68f46 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:54.131751 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8cc521-973c-4809-b32d-17932df68f46 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:54.133140 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8cc521-973c-4809-b32d-17932df68f46 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:54.154058 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 516/1066] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Jun 23 15:48:53 2022] POST /identity/v3/endpoints => generated 327 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 15:48:54.188159 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:54.236543 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638d846d-5e54-4968-a564-293bdf404e46 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:54.284750 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892585f2-2c1e-4035-94c6-12b3edcac1ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:54.285434 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892585f2-2c1e-4035-94c6-12b3edcac1ee None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:54.285810 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892585f2-2c1e-4035-94c6-12b3edcac1ee None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:54.285996 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892585f2-2c1e-4035-94c6-12b3edcac1ee None admin] PATH_INFO: `/v3/users` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:54.286887 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892585f2-2c1e-4035-94c6-12b3edcac1ee None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:48:54.287204 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892585f2-2c1e-4035-94c6-12b3edcac1ee None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:54.288696 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892585f2-2c1e-4035-94c6-12b3edcac1ee None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:54.309149 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 551/1067] 10.222.0.46 () {56 vars in 1133 bytes} [Thu Jun 23 15:48:54 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:48:54.314504 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8cc521-973c-4809-b32d-17932df68f46 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:54.401757 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:54.426317 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8cc521-973c-4809-b32d-17932df68f46 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:54.462759 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1de04cf-40f3-4990-a2a9-20169945e8d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:54.464080 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1de04cf-40f3-4990-a2a9-20169945e8d0 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:54.464609 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1de04cf-40f3-4990-a2a9-20169945e8d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:54.465065 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1de04cf-40f3-4990-a2a9-20169945e8d0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:54.478675 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1de04cf-40f3-4990-a2a9-20169945e8d0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:54.480486 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1de04cf-40f3-4990-a2a9-20169945e8d0 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:54.501026 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-b1de04cf-40f3-4990-a2a9-20169945e8d0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 15:48:54.501026 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:48:54.501026 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:48:54.501026 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:48:54.501026 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:48:54.501026 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:48:54.501026 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:48:54.501026 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:48:54.501026 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:48:54.501026 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:48:54.501026 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:48:54.501026 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:48:54.501026 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 15:48:54.501026 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 15:48:54.501026 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 15:48:54.501997 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 15:48:54.501997 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:48:54.501997 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:48:54.501997 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:48:54.501997 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:48:54.501997 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:48:54.501997 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:48:54.501997 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:48:54.501997 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Jun 23 15:48:54.501997 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:48:54.501997 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:48:54.501997 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:48:54.501997 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:48:54.501997 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:48:54.501997 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:48:54.501997 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:48:54.501997 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:48:54.503029 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 15:48:54.503029 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 15:48:54.503029 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 15:48:54.503029 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Jun 23 15:48:54.505524 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 517/1068] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Jun 23 15:48:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:48:54.511019 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892585f2-2c1e-4035-94c6-12b3edcac1ee None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:54.623433 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892585f2-2c1e-4035-94c6-12b3edcac1ee None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:54.642386 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:54.665963 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97e6d2c-7c95-4953-bc54-2a74b23c3d32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:54.666554 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97e6d2c-7c95-4953-bc54-2a74b23c3d32 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:54.666879 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97e6d2c-7c95-4953-bc54-2a74b23c3d32 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:54.667160 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97e6d2c-7c95-4953-bc54-2a74b23c3d32 None admin] PATH_INFO: `/v3/domains` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:54.667891 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97e6d2c-7c95-4953-bc54-2a74b23c3d32 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:48:54.668202 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97e6d2c-7c95-4953-bc54-2a74b23c3d32 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:54.669588 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97e6d2c-7c95-4953-bc54-2a74b23c3d32 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:54.686807 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 552/1069] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Jun 23 15:48:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:48:54.688434 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1de04cf-40f3-4990-a2a9-20169945e8d0 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:54.795559 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1de04cf-40f3-4990-a2a9-20169945e8d0 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:54.836339 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:54.844076 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2b8441-6567-4918-b386-37b0f3f6ca8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:54.844833 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2b8441-6567-4918-b386-37b0f3f6ca8d None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:54.845104 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2b8441-6567-4918-b386-37b0f3f6ca8d None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:54.845383 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2b8441-6567-4918-b386-37b0f3f6ca8d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:54.853062 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2b8441-6567-4918-b386-37b0f3f6ca8d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:54.854510 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2b8441-6567-4918-b386-37b0f3f6ca8d None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:54.865664 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b73d047-9ec2-4674-918c-6b38bda7d467 None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:54.865946 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b73d047-9ec2-4674-918c-6b38bda7d467 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:54.866232 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b73d047-9ec2-4674-918c-6b38bda7d467 None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:54.870729 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 553/1070] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jun 23 15:48:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 15:48:54.870885 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-ed2b8441-6567-4918-b386-37b0f3f6ca8d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 15:48:54.870885 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:48:54.870885 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:48:54.870885 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:48:54.870885 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:48:54.870885 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:48:54.870885 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:48:54.870885 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:48:54.870885 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:48:54.870885 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:48:54.870885 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:48:54.870885 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:48:54.870885 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 15:48:54.870885 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 15:48:54.870885 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 15:48:54.871717 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 15:48:54.871717 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:48:54.871717 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:48:54.871717 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:48:54.871717 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:48:54.871717 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:48:54.871717 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:48:54.871717 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:48:54.871717 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Jun 23 15:48:54.871717 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:48:54.871717 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:48:54.871717 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:48:54.871717 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:48:54.871717 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:48:54.871717 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:48:54.871717 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:48:54.871717 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:48:54.872607 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 15:48:54.872607 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 15:48:54.872607 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 15:48:54.872607 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 15:48:54.872607 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 15:48:54.872607 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 15:48:54.872607 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 15:48:54.872607 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Jun 23 15:48:54.872607 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 518/1071] 10.222.0.46 () {56 vars in 1111 bytes} [Thu Jun 23 15:48:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:48:54.877034 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b73d047-9ec2-4674-918c-6b38bda7d467 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:54.877661 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a0687b-ef30-4418-921c-9924e25123e5 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:54.877860 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a0687b-ef30-4418-921c-9924e25123e5 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:54.878176 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a0687b-ef30-4418-921c-9924e25123e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:54.895078 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-c3a0687b-ef30-4418-921c-9924e25123e5 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:48:54.896288 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-c3a0687b-ef30-4418-921c-9924e25123e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:48:54.975309 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 519/1072] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jun 23 15:48:54 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 23 15:48:54.998445 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a0687b-ef30-4418-921c-9924e25123e5 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:54.999452 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b73d047-9ec2-4674-918c-6b38bda7d467 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:55.044561 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea761de-c47b-4f08-b79e-c6cd19c4ba0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:55.045949 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea761de-c47b-4f08-b79e-c6cd19c4ba0d None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:55.046403 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea761de-c47b-4f08-b79e-c6cd19c4ba0d None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:55.046864 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea761de-c47b-4f08-b79e-c6cd19c4ba0d None admin] PATH_INFO: `/v3/projects` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:55.047788 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea761de-c47b-4f08-b79e-c6cd19c4ba0d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:48:55.048239 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea761de-c47b-4f08-b79e-c6cd19c4ba0d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:55.049901 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea761de-c47b-4f08-b79e-c6cd19c4ba0d None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:55.073289 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 554/1073] 10.222.0.46 () {56 vars in 1148 bytes} [Thu Jun 23 15:48:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 15:48:55.081570 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea761de-c47b-4f08-b79e-c6cd19c4ba0d None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:55.123188 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:55.146318 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a0687b-ef30-4418-921c-9924e25123e5 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:55.176033 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:55.192455 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4f5a37-9a25-42f0-8f1b-8d18ac5ceb66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:55.193345 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4f5a37-9a25-42f0-8f1b-8d18ac5ceb66 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:55.193630 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4f5a37-9a25-42f0-8f1b-8d18ac5ceb66 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:55.193900 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4f5a37-9a25-42f0-8f1b-8d18ac5ceb66 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:55.202776 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea761de-c47b-4f08-b79e-c6cd19c4ba0d None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:55.207741 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4f5a37-9a25-42f0-8f1b-8d18ac5ceb66 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 15:48:55.215861 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-4b4f5a37-9a25-42f0-8f1b-8d18ac5ceb66 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 15:48:55.215861 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:48:55.215861 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:48:55.215861 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:48:55.215861 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:48:55.215861 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:48:55.215861 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:48:55.215861 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:48:55.215861 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:48:55.215861 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:48:55.215861 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:48:55.215861 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:48:55.215861 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 15:48:55.215861 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 15:48:55.215861 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 15:48:55.215861 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 15:48:55.216829 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 15:48:55.216829 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 15:48:55.216829 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 15:48:55.216829 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 15:48:55.216829 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:48:55.216829 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:48:55.216829 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:48:55.216829 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:48:55.216829 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:48:55.216829 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:48:55.216829 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:48:55.216829 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Jun 23 15:48:55.216829 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:48:55.216829 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:48:55.216829 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:48:55.216829 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:48:55.217685 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:48:55.217685 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:48:55.217685 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:48:55.217685 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:48:55.217685 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 15:48:55.217685 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 15:48:55.217685 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 15:48:55.217685 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 15:48:55.217685 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 15:48:55.217685 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 15:48:55.217685 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 15:48:55.217685 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Jun 23 15:48:55.217685 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 520/1074] 10.222.0.46 () {56 vars in 1096 bytes} [Thu Jun 23 15:48:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 15:48:55.223790 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4f5a37-9a25-42f0-8f1b-8d18ac5ceb66 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:55.245841 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b14f47-5d34-4937-bd82-b51a910dcc0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:55.247081 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b14f47-5d34-4937-bd82-b51a910dcc0f None admin] REQUEST_METHOD: `PUT` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:55.247546 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b14f47-5d34-4937-bd82-b51a910dcc0f None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:55.248020 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b14f47-5d34-4937-bd82-b51a910dcc0f None admin] PATH_INFO: `/v3/projects/28ecdf5b474c44098c722f677d2a0b77/users/705115d19bab424d894bea8b1b48a7f0/roles/50fafe10bef649b4b87dfab0e16e7915` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:55.250277 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b14f47-5d34-4937-bd82-b51a910dcc0f None admin] RBAC: Authorizing `identity:create_grant(project_id=28ecdf5b474c44098c722f677d2a0b77, user_id=705115d19bab424d894bea8b1b48a7f0, role_id=50fafe10bef649b4b87dfab0e16e7915)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:55.251869 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b14f47-5d34-4937-bd82-b51a910dcc0f None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:55.278717 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 555/1075] 10.222.0.46 () {58 vars in 1439 bytes} [Thu Jun 23 15:48:55 2022] PUT /identity/v3/projects/28ecdf5b474c44098c722f677d2a0b77/users/705115d19bab424d894bea8b1b48a7f0/roles/50fafe10bef649b4b87dfab0e16e7915 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 15:48:55.349823 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4f5a37-9a25-42f0-8f1b-8d18ac5ceb66 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:55.389223 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4478157f-1278-460d-98f6-d0dc9ab5a079 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:55.389914 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4478157f-1278-460d-98f6-d0dc9ab5a079 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:55.390199 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4478157f-1278-460d-98f6-d0dc9ab5a079 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:55.390474 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4478157f-1278-460d-98f6-d0dc9ab5a079 None admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:55.391278 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4478157f-1278-460d-98f6-d0dc9ab5a079 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:48:55.391589 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4478157f-1278-460d-98f6-d0dc9ab5a079 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:55.393036 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4478157f-1278-460d-98f6-d0dc9ab5a079 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:55.405955 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 521/1076] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Jun 23 15:48:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 15:48:55.410114 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b14f47-5d34-4937-bd82-b51a910dcc0f None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:55.517864 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:55.526214 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b14f47-5d34-4937-bd82-b51a910dcc0f None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:55.565090 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6965c430-264f-496c-bdbc-c3019e8c02ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:55.566489 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6965c430-264f-496c-bdbc-c3019e8c02ec None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:55.566791 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6965c430-264f-496c-bdbc-c3019e8c02ec None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:55.567063 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6965c430-264f-496c-bdbc-c3019e8c02ec None admin] PATH_INFO: `/v3/users/d86946037a1d41ea943b1b598457b267` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:55.568243 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6965c430-264f-496c-bdbc-c3019e8c02ec None admin] RBAC: Authorizing `identity:get_user(user_id=d86946037a1d41ea943b1b598457b267)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:55.569675 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6965c430-264f-496c-bdbc-c3019e8c02ec None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:55.574005 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 556/1077] 10.222.0.46 () {56 vars in 1177 bytes} [Thu Jun 23 15:48:55 2022] GET /identity/v3/users/d86946037a1d41ea943b1b598457b267 => generated 285 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 15:48:55.577555 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4478157f-1278-460d-98f6-d0dc9ab5a079 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:55.624121 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:55.684917 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4478157f-1278-460d-98f6-d0dc9ab5a079 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:55.720255 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe489c8-f828-4da8-86fa-94f04419e4f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:55.721134 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe489c8-f828-4da8-86fa-94f04419e4f6 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:55.721419 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe489c8-f828-4da8-86fa-94f04419e4f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:55.721725 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe489c8-f828-4da8-86fa-94f04419e4f6 None admin] PATH_INFO: `/v3/projects/68a2c61ef29c405e9509ad7cc3121def` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:55.722958 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe489c8-f828-4da8-86fa-94f04419e4f6 None admin] RBAC: Authorizing `identity:get_project(project_id=68a2c61ef29c405e9509ad7cc3121def)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:55.724397 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe489c8-f828-4da8-86fa-94f04419e4f6 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:55.726263 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 522/1078] 10.222.0.46 () {56 vars in 1186 bytes} [Thu Jun 23 15:48:55 2022] GET /identity/v3/projects/68a2c61ef29c405e9509ad7cc3121def => generated 311 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:48:55.732049 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6965c430-264f-496c-bdbc-c3019e8c02ec None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:55.843985 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6965c430-264f-496c-bdbc-c3019e8c02ec None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:55.877990 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b694c0-5999-4066-bb1a-ed68679ecb1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:55.879623 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b694c0-5999-4066-bb1a-ed68679ecb1d None admin] REQUEST_METHOD: `PUT` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:55.879913 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b694c0-5999-4066-bb1a-ed68679ecb1d None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:55.880205 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b694c0-5999-4066-bb1a-ed68679ecb1d None admin] PATH_INFO: `/v3/projects/68a2c61ef29c405e9509ad7cc3121def/users/d86946037a1d41ea943b1b598457b267/roles/812d3c567c52469cb2b35709fbdc857c` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:55.882230 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b694c0-5999-4066-bb1a-ed68679ecb1d None admin] RBAC: Authorizing `identity:create_grant(project_id=68a2c61ef29c405e9509ad7cc3121def, user_id=d86946037a1d41ea943b1b598457b267, role_id=812d3c567c52469cb2b35709fbdc857c)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:55.883722 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b694c0-5999-4066-bb1a-ed68679ecb1d None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:55.897483 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 557/1079] 10.222.0.46 () {58 vars in 1439 bytes} [Thu Jun 23 15:48:55 2022] PUT /identity/v3/projects/68a2c61ef29c405e9509ad7cc3121def/users/d86946037a1d41ea943b1b598457b267/roles/812d3c567c52469cb2b35709fbdc857c => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 15:48:56.245239 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de860ef-0840-4d3a-ac7b-50cd0d9852d5 None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:56.245589 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de860ef-0840-4d3a-ac7b-50cd0d9852d5 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:56.245707 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de860ef-0840-4d3a-ac7b-50cd0d9852d5 None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:56.247293 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 523/1080] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jun 23 15:48:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 15:48:56.252371 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8779c4-2617-46d4-a309-046f67b0fcc7 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:56.252881 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8779c4-2617-46d4-a309-046f67b0fcc7 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:56.253043 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8779c4-2617-46d4-a309-046f67b0fcc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:56.270836 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-8d8779c4-2617-46d4-a309-046f67b0fcc7 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:48:56.272098 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-8d8779c4-2617-46d4-a309-046f67b0fcc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:48:56.292039 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:56.350712 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 558/1081] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jun 23 15:48:56 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 15:48:56.373882 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de860ef-0840-4d3a-ac7b-50cd0d9852d5 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:56.480519 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de860ef-0840-4d3a-ac7b-50cd0d9852d5 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:56.491315 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:56.515720 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f077d981-78de-462b-830d-c03299018255 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:56.516301 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f077d981-78de-462b-830d-c03299018255 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:56.516586 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f077d981-78de-462b-830d-c03299018255 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:56.516864 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f077d981-78de-462b-830d-c03299018255 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:56.598323 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f077d981-78de-462b-830d-c03299018255 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:56.599812 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f077d981-78de-462b-830d-c03299018255 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:56.700436 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 524/1082] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Jun 23 15:48:56 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 23 15:48:56.706649 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8779c4-2617-46d4-a309-046f67b0fcc7 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:56.807126 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8779c4-2617-46d4-a309-046f67b0fcc7 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:56.841589 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6532c97-9f9e-4123-8c08-fe82203615ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:56.842396 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6532c97-9f9e-4123-8c08-fe82203615ca None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:56.842677 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6532c97-9f9e-4123-8c08-fe82203615ca None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:56.842973 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6532c97-9f9e-4123-8c08-fe82203615ca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:56.853949 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6532c97-9f9e-4123-8c08-fe82203615ca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:56.855468 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6532c97-9f9e-4123-8c08-fe82203615ca None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:56.873001 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-e6532c97-9f9e-4123-8c08-fe82203615ca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 15:48:56.873001 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:48:56.873001 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:48:56.873001 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:48:56.873001 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:48:56.873001 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:48:56.873001 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:48:56.873001 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:48:56.873001 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:48:56.873001 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:48:56.873001 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:48:56.873001 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:48:56.873001 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 15:48:56.873001 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 15:48:56.873001 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 15:48:56.873943 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 15:48:56.873943 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:48:56.873943 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:48:56.873943 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:48:56.873943 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:48:56.873943 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:48:56.873943 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:48:56.873943 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:48:56.873943 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Jun 23 15:48:56.873943 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:48:56.873943 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:48:56.873943 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:48:56.873943 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:48:56.873943 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:48:56.873943 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:48:56.873943 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:48:56.873943 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:48:56.875277 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 15:48:56.875277 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 15:48:56.875277 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 15:48:56.875277 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Jun 23 15:48:56.878776 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 559/1083] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Jun 23 15:48:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:48:56.880674 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f077d981-78de-462b-830d-c03299018255 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:56.892486 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:56.981819 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f077d981-78de-462b-830d-c03299018255 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:57.019845 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92261416-b949-4696-8c67-b23ec163e7fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:57.020658 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92261416-b949-4696-8c67-b23ec163e7fc None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:57.020942 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92261416-b949-4696-8c67-b23ec163e7fc None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:57.021218 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92261416-b949-4696-8c67-b23ec163e7fc None admin] PATH_INFO: `/v3/domains` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:57.021947 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92261416-b949-4696-8c67-b23ec163e7fc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:48:57.022262 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92261416-b949-4696-8c67-b23ec163e7fc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:57.023675 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92261416-b949-4696-8c67-b23ec163e7fc None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:57.042737 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 525/1084] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Jun 23 15:48:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 15:48:57.315988 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0efeb7-a145-4c19-8c47-abc82d83be0d None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:57.316508 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0efeb7-a145-4c19-8c47-abc82d83be0d None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:57.316629 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0efeb7-a145-4c19-8c47-abc82d83be0d None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:57.317921 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 560/1085] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jun 23 15:48:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 15:48:57.323245 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87687cc3-a4bd-4785-ba79-8108d8b3a82e None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:57.323544 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87687cc3-a4bd-4785-ba79-8108d8b3a82e None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:57.323830 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87687cc3-a4bd-4785-ba79-8108d8b3a82e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:57.340354 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-87687cc3-a4bd-4785-ba79-8108d8b3a82e None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:48:57.341550 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-87687cc3-a4bd-4785-ba79-8108d8b3a82e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:48:57.406147 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:57.412330 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 526/1086] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jun 23 15:48:57 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 15:48:57.435397 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0efeb7-a145-4c19-8c47-abc82d83be0d None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:57.445001 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:57.541030 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0efeb7-a145-4c19-8c47-abc82d83be0d None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:57.578662 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3757b75-2536-4add-bd3e-e57b249c5bac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:57.579511 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3757b75-2536-4add-bd3e-e57b249c5bac None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:57.579794 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3757b75-2536-4add-bd3e-e57b249c5bac None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:57.580072 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3757b75-2536-4add-bd3e-e57b249c5bac None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:57.590299 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3757b75-2536-4add-bd3e-e57b249c5bac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 15:48:57.598487 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-a3757b75-2536-4add-bd3e-e57b249c5bac None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 23 15:48:57.598487 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:48:57.598487 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:48:57.598487 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:48:57.598487 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:48:57.598487 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:48:57.598487 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:48:57.598487 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:48:57.598487 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:48:57.598487 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:48:57.598487 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:48:57.598487 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:48:57.598487 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 15:48:57.598487 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 15:48:57.598487 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 15:48:57.598487 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 15:48:57.599611 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 15:48:57.599611 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 15:48:57.599611 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 15:48:57.599611 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 15:48:57.599611 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:48:57.599611 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:48:57.599611 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:48:57.599611 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:48:57.599611 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:48:57.599611 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:48:57.599611 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:48:57.599611 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Jun 23 15:48:57.599611 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:48:57.599611 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:48:57.599611 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:48:57.599611 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:48:57.600856 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:48:57.600856 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:48:57.600856 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:48:57.600856 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:48:57.600856 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 15:48:57.600856 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 15:48:57.600856 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 15:48:57.600856 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 15:48:57.600856 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 15:48:57.600856 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 15:48:57.600856 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 23 15:48:57.600856 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Jun 23 15:48:57.600856 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 561/1087] 10.222.0.46 () {56 vars in 1120 bytes} [Thu Jun 23 15:48:57 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:48:57.605650 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87687cc3-a4bd-4785-ba79-8108d8b3a82e None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:57.703761 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87687cc3-a4bd-4785-ba79-8108d8b3a82e None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:57.738356 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebda9945-2cff-4802-b1e2-684a9a84ff36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:57.739069 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebda9945-2cff-4802-b1e2-684a9a84ff36 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:57.739352 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebda9945-2cff-4802-b1e2-684a9a84ff36 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:57.739630 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebda9945-2cff-4802-b1e2-684a9a84ff36 None admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:57.740380 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebda9945-2cff-4802-b1e2-684a9a84ff36 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:48:57.740691 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebda9945-2cff-4802-b1e2-684a9a84ff36 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:57.742058 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebda9945-2cff-4802-b1e2-684a9a84ff36 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:57.750077 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 527/1088] 10.222.0.46 () {56 vars in 1115 bytes} [Thu Jun 23 15:48:57 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:48:57.755775 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3757b75-2536-4add-bd3e-e57b249c5bac None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:57.820322 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:57.854914 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3757b75-2536-4add-bd3e-e57b249c5bac None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:57.888131 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098a7dbc-d8cb-49b0-a5cd-514d253150e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:57.888925 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098a7dbc-d8cb-49b0-a5cd-514d253150e1 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:57.889209 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098a7dbc-d8cb-49b0-a5cd-514d253150e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:57.889481 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098a7dbc-d8cb-49b0-a5cd-514d253150e1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:57.900281 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098a7dbc-d8cb-49b0-a5cd-514d253150e1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:57.901725 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098a7dbc-d8cb-49b0-a5cd-514d253150e1 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:57.919080 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-098a7dbc-d8cb-49b0-a5cd-514d253150e1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 15:48:57.919080 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:48:57.919080 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:48:57.919080 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:48:57.919080 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:48:57.919080 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:48:57.919080 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:48:57.919080 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:48:57.919080 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:48:57.919080 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:48:57.919080 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:48:57.919080 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:48:57.919080 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 15:48:57.919080 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 15:48:57.919080 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 15:48:57.919974 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 15:48:57.919974 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:48:57.919974 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:48:57.919974 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:48:57.919974 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:48:57.919974 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:48:57.919974 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:48:57.919974 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:48:57.919974 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Jun 23 15:48:57.919974 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:48:57.919974 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:48:57.919974 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:48:57.919974 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:48:57.919974 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:48:57.919974 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:48:57.919974 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:48:57.919974 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:48:57.920865 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 15:48:57.920865 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 15:48:57.920865 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 15:48:57.920865 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Jun 23 15:48:57.920865 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 562/1089] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Jun 23 15:48:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:48:57.926531 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebda9945-2cff-4802-b1e2-684a9a84ff36 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:57.938543 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4ab4b8-7227-4519-ba58-f584ae6e028a None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:57.938906 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4ab4b8-7227-4519-ba58-f584ae6e028a None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:57.939198 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4ab4b8-7227-4519-ba58-f584ae6e028a None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:57.940564 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 563/1090] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jun 23 15:48:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 15:48:57.945487 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbfda8d-a70a-4dfe-8947-bd53f83258f5 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:57.945772 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbfda8d-a70a-4dfe-8947-bd53f83258f5 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:57.946058 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbfda8d-a70a-4dfe-8947-bd53f83258f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:57.962828 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-8fbfda8d-a70a-4dfe-8947-bd53f83258f5 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:48:57.964022 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-8fbfda8d-a70a-4dfe-8947-bd53f83258f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:48:57.989902 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:58.029126 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebda9945-2cff-4802-b1e2-684a9a84ff36 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:58.031160 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 564/1091] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jun 23 15:48:57 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 23 15:48:58.055350 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbfda8d-a70a-4dfe-8947-bd53f83258f5 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:58.066199 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5e5243-268e-4a52-a380-bf108aaf3591 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:58.066910 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5e5243-268e-4a52-a380-bf108aaf3591 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:58.067197 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5e5243-268e-4a52-a380-bf108aaf3591 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:58.067477 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5e5243-268e-4a52-a380-bf108aaf3591 None admin] PATH_INFO: `/v3/domains` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:58.068201 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5e5243-268e-4a52-a380-bf108aaf3591 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:48:58.068508 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5e5243-268e-4a52-a380-bf108aaf3591 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:58.069913 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5e5243-268e-4a52-a380-bf108aaf3591 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:58.082172 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 528/1092] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Jun 23 15:48:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 15:48:58.087762 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5e5243-268e-4a52-a380-bf108aaf3591 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:58.103683 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:58.155554 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbfda8d-a70a-4dfe-8947-bd53f83258f5 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:58.185928 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5e5243-268e-4a52-a380-bf108aaf3591 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:58.189566 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8236bbb2-21f6-4a7a-96dc-ffdfd51f4fd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:58.190364 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8236bbb2-21f6-4a7a-96dc-ffdfd51f4fd1 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:58.190651 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8236bbb2-21f6-4a7a-96dc-ffdfd51f4fd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:58.190954 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8236bbb2-21f6-4a7a-96dc-ffdfd51f4fd1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:58.201088 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8236bbb2-21f6-4a7a-96dc-ffdfd51f4fd1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 15:48:58.209110 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-8236bbb2-21f6-4a7a-96dc-ffdfd51f4fd1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 15:48:58.209110 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:48:58.209110 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:48:58.209110 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:48:58.209110 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:48:58.209110 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:48:58.209110 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:48:58.209110 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:48:58.209110 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:48:58.209110 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:48:58.209110 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:48:58.209110 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:48:58.209110 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 15:48:58.209110 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 15:48:58.209110 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 15:48:58.209110 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 15:48:58.210134 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 15:48:58.210134 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 15:48:58.210134 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 15:48:58.210134 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 15:48:58.210134 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:48:58.210134 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:48:58.210134 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:48:58.210134 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:48:58.210134 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:48:58.210134 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:48:58.210134 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:48:58.210134 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Jun 23 15:48:58.210134 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:48:58.210134 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:48:58.210134 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:48:58.210134 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:48:58.211139 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:48:58.211139 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:48:58.211139 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:48:58.211139 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:48:58.211139 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 15:48:58.211139 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 15:48:58.211139 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 15:48:58.211139 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 15:48:58.211139 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 15:48:58.211139 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 15:48:58.211139 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 15:48:58.211139 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Jun 23 15:48:58.211898 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:58.211983 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 565/1093] 10.222.0.46 () {56 vars in 1096 bytes} [Thu Jun 23 15:48:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 15:48:58.216705 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8236bbb2-21f6-4a7a-96dc-ffdfd51f4fd1 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:58.222396 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64dbe8bb-0078-45bd-a7c2-2f3de8b8f270 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:58.223232 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dbe8bb-0078-45bd-a7c2-2f3de8b8f270 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:58.223517 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dbe8bb-0078-45bd-a7c2-2f3de8b8f270 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:58.223791 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dbe8bb-0078-45bd-a7c2-2f3de8b8f270 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:58.231663 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64dbe8bb-0078-45bd-a7c2-2f3de8b8f270 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:58.233105 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64dbe8bb-0078-45bd-a7c2-2f3de8b8f270 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:58.247867 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-64dbe8bb-0078-45bd-a7c2-2f3de8b8f270 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 23 15:48:58.247867 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:48:58.247867 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:48:58.247867 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:48:58.247867 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:48:58.247867 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:48:58.247867 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:48:58.247867 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:48:58.247867 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:48:58.247867 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:48:58.247867 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:48:58.247867 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:48:58.247867 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 15:48:58.247867 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 15:48:58.247867 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 15:48:58.247867 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 15:48:58.249060 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:48:58.249060 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:48:58.249060 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 15:48:58.249060 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 15:48:58.249060 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 15:48:58.249060 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 15:48:58.249060 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:48:58.249060 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:48:58.249060 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:48:58.249060 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:48:58.249060 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:48:58.249060 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Jun 23 15:48:58.249060 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:48:58.249060 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:48:58.249060 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:48:58.249060 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:48:58.249060 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:48:58.249060 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:48:58.250483 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:48:58.250483 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:48:58.250483 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 15:48:58.250483 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 15:48:58.250483 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 15:48:58.250483 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 15:48:58.250483 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 15:48:58.250483 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 15:48:58.250483 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 23 15:48:58.250483 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Jun 23 15:48:58.250483 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 529/1094] 10.222.0.46 () {56 vars in 1093 bytes} [Thu Jun 23 15:48:58 2022] GET /identity/v3/users/nova => generated 82 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 15:48:58.254766 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64dbe8bb-0078-45bd-a7c2-2f3de8b8f270 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:58.320498 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8236bbb2-21f6-4a7a-96dc-ffdfd51f4fd1 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:58.329346 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:58.353409 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64dbe8bb-0078-45bd-a7c2-2f3de8b8f270 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:58.355276 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251ff43a-9766-4656-8959-adcbd720eeb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:58.355953 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251ff43a-9766-4656-8959-adcbd720eeb9 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:58.356237 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251ff43a-9766-4656-8959-adcbd720eeb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:58.356513 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251ff43a-9766-4656-8959-adcbd720eeb9 None admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:58.357272 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251ff43a-9766-4656-8959-adcbd720eeb9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:48:58.357575 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251ff43a-9766-4656-8959-adcbd720eeb9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:58.358976 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251ff43a-9766-4656-8959-adcbd720eeb9 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:58.367090 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 566/1095] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Jun 23 15:48:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 15:48:58.372730 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251ff43a-9766-4656-8959-adcbd720eeb9 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:58.389422 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0e93f2-5991-47d3-bc60-fa8b29eba8f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:58.390103 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0e93f2-5991-47d3-bc60-fa8b29eba8f2 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:58.390393 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0e93f2-5991-47d3-bc60-fa8b29eba8f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:58.390672 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0e93f2-5991-47d3-bc60-fa8b29eba8f2 None admin] PATH_INFO: `/v3/users` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:58.391562 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0e93f2-5991-47d3-bc60-fa8b29eba8f2 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:48:58.391875 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0e93f2-5991-47d3-bc60-fa8b29eba8f2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:58.393318 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0e93f2-5991-47d3-bc60-fa8b29eba8f2 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:58.416368 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 530/1096] 10.222.0.46 () {56 vars in 1133 bytes} [Thu Jun 23 15:48:58 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 15:48:58.417356 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0e93f2-5991-47d3-bc60-fa8b29eba8f2 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:58.457739 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:58.476447 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251ff43a-9766-4656-8959-adcbd720eeb9 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:58.509610 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7317c301-331d-4b9c-8d5e-39383cad2c77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:58.510489 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7317c301-331d-4b9c-8d5e-39383cad2c77 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:58.510869 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7317c301-331d-4b9c-8d5e-39383cad2c77 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:58.511150 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7317c301-331d-4b9c-8d5e-39383cad2c77 None admin] PATH_INFO: `/v3/users/d86946037a1d41ea943b1b598457b267` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:58.512342 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7317c301-331d-4b9c-8d5e-39383cad2c77 None admin] RBAC: Authorizing `identity:get_user(user_id=d86946037a1d41ea943b1b598457b267)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:58.513786 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7317c301-331d-4b9c-8d5e-39383cad2c77 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:58.515489 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 567/1097] 10.222.0.46 () {56 vars in 1177 bytes} [Thu Jun 23 15:48:58 2022] GET /identity/v3/users/d86946037a1d41ea943b1b598457b267 => generated 285 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 15:48:58.516141 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0e93f2-5991-47d3-bc60-fa8b29eba8f2 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:58.521703 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7317c301-331d-4b9c-8d5e-39383cad2c77 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:58.550903 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babc8c61-a158-40e7-8a3c-c11e7352b80f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:58.551577 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babc8c61-a158-40e7-8a3c-c11e7352b80f None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:58.551879 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babc8c61-a158-40e7-8a3c-c11e7352b80f None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:58.552161 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babc8c61-a158-40e7-8a3c-c11e7352b80f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:58.564033 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:58.621641 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babc8c61-a158-40e7-8a3c-c11e7352b80f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:58.623169 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babc8c61-a158-40e7-8a3c-c11e7352b80f None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:58.624585 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7317c301-331d-4b9c-8d5e-39383cad2c77 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:58.659475 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d126d635-7f17-45e2-933f-dda1bdc8f886 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:58.660128 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d126d635-7f17-45e2-933f-dda1bdc8f886 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:58.660423 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d126d635-7f17-45e2-933f-dda1bdc8f886 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:58.660694 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d126d635-7f17-45e2-933f-dda1bdc8f886 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:58.667998 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:58.726415 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 531/1098] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Jun 23 15:48:58 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 23 15:48:58.728267 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d126d635-7f17-45e2-933f-dda1bdc8f886 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:58.729855 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d126d635-7f17-45e2-933f-dda1bdc8f886 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:58.732183 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babc8c61-a158-40e7-8a3c-c11e7352b80f None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:58.769223 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:58.830300 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babc8c61-a158-40e7-8a3c-c11e7352b80f None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:58.832252 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 568/1099] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Jun 23 15:48:58 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 23 15:48:58.838066 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d126d635-7f17-45e2-933f-dda1bdc8f886 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:58.869654 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d7b041-3204-44ca-9acb-cb55c6a3087c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:58.870483 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d7b041-3204-44ca-9acb-cb55c6a3087c None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:58.870800 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d7b041-3204-44ca-9acb-cb55c6a3087c None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:58.871079 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d7b041-3204-44ca-9acb-cb55c6a3087c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:58.879807 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:58.882027 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d7b041-3204-44ca-9acb-cb55c6a3087c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:58.884031 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d7b041-3204-44ca-9acb-cb55c6a3087c None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:58.901743 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-a9d7b041-3204-44ca-9acb-cb55c6a3087c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 15:48:58.901743 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:48:58.901743 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:48:58.901743 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:48:58.901743 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:48:58.901743 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:48:58.901743 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:48:58.901743 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:48:58.901743 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:48:58.901743 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:48:58.901743 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:48:58.901743 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:48:58.901743 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 15:48:58.901743 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 15:48:58.901743 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 15:48:58.902653 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 15:48:58.902653 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:48:58.902653 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:48:58.902653 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:48:58.902653 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:48:58.902653 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:48:58.902653 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:48:58.902653 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:48:58.902653 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Jun 23 15:48:58.902653 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:48:58.902653 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:48:58.902653 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:48:58.902653 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:48:58.902653 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:48:58.902653 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:48:58.902653 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:48:58.902653 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:48:58.903606 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 15:48:58.903606 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 15:48:58.903606 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 15:48:58.903606 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Jun 23 15:48:58.903606 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 532/1100] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Jun 23 15:48:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 15:48:58.908499 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d7b041-3204-44ca-9acb-cb55c6a3087c None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:58.942197 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d126d635-7f17-45e2-933f-dda1bdc8f886 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:58.975263 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8d2bb0-beda-4425-9205-f0711d17f50e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:58.976128 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8d2bb0-beda-4425-9205-f0711d17f50e None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:58.976413 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8d2bb0-beda-4425-9205-f0711d17f50e None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:58.976694 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8d2bb0-beda-4425-9205-f0711d17f50e None admin] PATH_INFO: `/v3/projects/68a2c61ef29c405e9509ad7cc3121def` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:58.977882 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8d2bb0-beda-4425-9205-f0711d17f50e None admin] RBAC: Authorizing `identity:get_project(project_id=68a2c61ef29c405e9509ad7cc3121def)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:58.979356 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8d2bb0-beda-4425-9205-f0711d17f50e None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:58.981064 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 569/1101] 10.222.0.46 () {56 vars in 1186 bytes} [Thu Jun 23 15:48:58 2022] GET /identity/v3/projects/68a2c61ef29c405e9509ad7cc3121def => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 15:48:58.987593 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8d2bb0-beda-4425-9205-f0711d17f50e None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:58.995766 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:59.008101 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d7b041-3204-44ca-9acb-cb55c6a3087c None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:59.042564 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e158c319-2542-45e8-b6df-28969b0c0b0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:59.043261 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e158c319-2542-45e8-b6df-28969b0c0b0f None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:59.043548 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e158c319-2542-45e8-b6df-28969b0c0b0f None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:59.043831 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e158c319-2542-45e8-b6df-28969b0c0b0f None admin] PATH_INFO: `/v3/domains` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:59.044557 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e158c319-2542-45e8-b6df-28969b0c0b0f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:48:59.044868 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e158c319-2542-45e8-b6df-28969b0c0b0f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:59.046241 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e158c319-2542-45e8-b6df-28969b0c0b0f None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:59.058044 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 533/1102] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Jun 23 15:48:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 15:48:59.063387 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e158c319-2542-45e8-b6df-28969b0c0b0f None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:59.089130 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8d2bb0-beda-4425-9205-f0711d17f50e None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:59.116801 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:59.123405 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1b013f-3cdf-490a-92dc-0ad202d746e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:59.124066 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1b013f-3cdf-490a-92dc-0ad202d746e1 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:59.124349 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1b013f-3cdf-490a-92dc-0ad202d746e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:59.124632 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1b013f-3cdf-490a-92dc-0ad202d746e1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:59.125464 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1b013f-3cdf-490a-92dc-0ad202d746e1 None admin] RBAC: Adding query filter params (role.id=812d3c567c52469cb2b35709fbdc857c, scope.project.id=68a2c61ef29c405e9509ad7cc3121def, user.id=d86946037a1d41ea943b1b598457b267) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:48:59.125779 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1b013f-3cdf-490a-92dc-0ad202d746e1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:59.127254 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1b013f-3cdf-490a-92dc-0ad202d746e1 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:59.133651 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 570/1103] 10.222.0.46 () {56 vars in 1374 bytes} [Thu Jun 23 15:48:58 2022] GET /identity/v3/role_assignments?user.id=d86946037a1d41ea943b1b598457b267&scope.project.id=68a2c61ef29c405e9509ad7cc3121def&role.id=812d3c567c52469cb2b35709fbdc857c => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 15:48:59.169091 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e158c319-2542-45e8-b6df-28969b0c0b0f None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:59.210259 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca70979-df91-423f-9ed4-81382be65008 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:59.211100 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca70979-df91-423f-9ed4-81382be65008 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:59.211385 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca70979-df91-423f-9ed4-81382be65008 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:59.211787 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca70979-df91-423f-9ed4-81382be65008 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:59.220105 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca70979-df91-423f-9ed4-81382be65008 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:59.222119 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca70979-df91-423f-9ed4-81382be65008 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:59.238434 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-aca70979-df91-423f-9ed4-81382be65008 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 15:48:59.238434 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:48:59.238434 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:48:59.238434 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:48:59.238434 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:48:59.238434 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:48:59.238434 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:48:59.238434 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:48:59.238434 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:48:59.238434 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:48:59.238434 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:48:59.238434 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:48:59.238434 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 15:48:59.238434 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 15:48:59.238434 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 15:48:59.239546 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 15:48:59.239546 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:48:59.239546 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:48:59.239546 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:48:59.239546 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:48:59.239546 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:48:59.239546 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:48:59.239546 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:48:59.239546 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Jun 23 15:48:59.239546 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:48:59.239546 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:48:59.239546 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:48:59.239546 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:48:59.239546 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:48:59.239546 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:48:59.239546 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:48:59.239546 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:48:59.240460 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 15:48:59.240460 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 15:48:59.240460 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 15:48:59.240460 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 15:48:59.240460 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 15:48:59.240460 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 15:48:59.240460 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 15:48:59.240460 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Jun 23 15:48:59.240460 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 534/1104] 10.222.0.46 () {56 vars in 1111 bytes} [Thu Jun 23 15:48:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 15:48:59.245690 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1b013f-3cdf-490a-92dc-0ad202d746e1 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:59.321188 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:59.355835 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1b013f-3cdf-490a-92dc-0ad202d746e1 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:59.392756 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10a2f19-c96a-486d-bb1d-f20e70711df5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:59.393472 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10a2f19-c96a-486d-bb1d-f20e70711df5 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:59.393777 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10a2f19-c96a-486d-bb1d-f20e70711df5 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:59.394085 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10a2f19-c96a-486d-bb1d-f20e70711df5 None admin] PATH_INFO: `/v3/projects` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:59.394939 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10a2f19-c96a-486d-bb1d-f20e70711df5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:48:59.395256 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10a2f19-c96a-486d-bb1d-f20e70711df5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:59.396710 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10a2f19-c96a-486d-bb1d-f20e70711df5 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:59.410179 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 571/1105] 10.222.0.46 () {56 vars in 1148 bytes} [Thu Jun 23 15:48:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:48:59.416452 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca70979-df91-423f-9ed4-81382be65008 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:48:59.527048 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca70979-df91-423f-9ed4-81382be65008 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:48:59.541982 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:48:59.568594 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8f32e9-c70c-4dad-a8b4-2a52cce5acb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:48:59.569273 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8f32e9-c70c-4dad-a8b4-2a52cce5acb7 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:48:59.569563 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8f32e9-c70c-4dad-a8b4-2a52cce5acb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:48:59.569841 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8f32e9-c70c-4dad-a8b4-2a52cce5acb7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:48:59.570721 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8f32e9-c70c-4dad-a8b4-2a52cce5acb7 None admin] RBAC: Adding query filter params (role.id=50fafe10bef649b4b87dfab0e16e7915, scope.project.id=28ecdf5b474c44098c722f677d2a0b77, user.id=705115d19bab424d894bea8b1b48a7f0) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:48:59.571021 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8f32e9-c70c-4dad-a8b4-2a52cce5acb7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:48:59.572507 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8f32e9-c70c-4dad-a8b4-2a52cce5acb7 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:48:59.579109 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 535/1106] 10.222.0.46 () {56 vars in 1374 bytes} [Thu Jun 23 15:48:59 2022] GET /identity/v3/role_assignments?user.id=705115d19bab424d894bea8b1b48a7f0&scope.project.id=28ecdf5b474c44098c722f677d2a0b77&role.id=50fafe10bef649b4b87dfab0e16e7915 => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:49:01.244738 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deef0cd8-fbef-4334-9b68-4b57d06af44d None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:49:01.245318 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deef0cd8-fbef-4334-9b68-4b57d06af44d None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:49:01.245318 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deef0cd8-fbef-4334-9b68-4b57d06af44d None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:49:01.246640 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 572/1107] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jun 23 15:49:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 15:49:01.251697 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1e651f-7c9e-4cc1-9ceb-6724b2456f42 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:49:01.252102 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1e651f-7c9e-4cc1-9ceb-6724b2456f42 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:49:01.252244 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1e651f-7c9e-4cc1-9ceb-6724b2456f42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:49:01.269328 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-bd1e651f-7c9e-4cc1-9ceb-6724b2456f42 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:49:01.270509 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-bd1e651f-7c9e-4cc1-9ceb-6724b2456f42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:49:01.334415 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 536/1108] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jun 23 15:49:01 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 23 15:49:01.358886 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deef0cd8-fbef-4334-9b68-4b57d06af44d None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:49:01.459367 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deef0cd8-fbef-4334-9b68-4b57d06af44d None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:49:01.487958 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:49:01.495005 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7344a7bd-2ed5-4c20-921e-0fcdf5a9786e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:49:01.495747 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7344a7bd-2ed5-4c20-921e-0fcdf5a9786e None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:49:01.496034 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7344a7bd-2ed5-4c20-921e-0fcdf5a9786e None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:49:01.496313 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7344a7bd-2ed5-4c20-921e-0fcdf5a9786e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:49:01.497492 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7344a7bd-2ed5-4c20-921e-0fcdf5a9786e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:49:01.498972 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7344a7bd-2ed5-4c20-921e-0fcdf5a9786e None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:49:01.500609 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 573/1109] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Jun 23 15:49:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:49:01.507811 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1e651f-7c9e-4cc1-9ceb-6724b2456f42 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:49:01.610325 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1e651f-7c9e-4cc1-9ceb-6724b2456f42 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:49:01.618044 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:49:01.644279 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97b9e3c-8497-4ff7-8501-3e05717e4872 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:49:01.644943 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97b9e3c-8497-4ff7-8501-3e05717e4872 None admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:49:01.645225 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97b9e3c-8497-4ff7-8501-3e05717e4872 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:49:01.645497 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97b9e3c-8497-4ff7-8501-3e05717e4872 None admin] PATH_INFO: `/v3/users` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:49:01.646210 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97b9e3c-8497-4ff7-8501-3e05717e4872 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:49:01.647672 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97b9e3c-8497-4ff7-8501-3e05717e4872 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:49:01.668167 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 537/1110] 10.222.0.46 () {60 vars in 1132 bytes} [Thu Jun 23 15:49:01 2022] POST /identity/v3/users => generated 286 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 15:49:03.726393 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e871b350-2d80-4c1c-9715-898d1cfe73a6 None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:49:03.726967 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e871b350-2d80-4c1c-9715-898d1cfe73a6 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:49:03.726967 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e871b350-2d80-4c1c-9715-898d1cfe73a6 None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:49:03.728331 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 574/1111] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jun 23 15:49:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 15:49:03.733557 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d18492-c880-4bac-b903-4e787e69b411 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:49:03.733945 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d18492-c880-4bac-b903-4e787e69b411 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:49:03.734112 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d18492-c880-4bac-b903-4e787e69b411 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:49:03.750440 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-91d18492-c880-4bac-b903-4e787e69b411 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:49:03.751663 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-91d18492-c880-4bac-b903-4e787e69b411 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:49:03.815866 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 538/1112] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jun 23 15:49:03 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 15:49:03.839672 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e871b350-2d80-4c1c-9715-898d1cfe73a6 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:49:03.942051 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e871b350-2d80-4c1c-9715-898d1cfe73a6 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:49:03.976360 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19a2bfe-2657-47e3-a040-fcc1916a4fdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:49:03.977228 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19a2bfe-2657-47e3-a040-fcc1916a4fdb None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:49:03.977512 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19a2bfe-2657-47e3-a040-fcc1916a4fdb None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:49:03.977789 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19a2bfe-2657-47e3-a040-fcc1916a4fdb None admin] PATH_INFO: `/v3/roles/d4d557a3ec4e434f8e83499c5fcc7136` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:49:03.979300 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19a2bfe-2657-47e3-a040-fcc1916a4fdb None admin] RBAC: Authorizing `identity:get_role(role_id=d4d557a3ec4e434f8e83499c5fcc7136)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:49:03.980691 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19a2bfe-2657-47e3-a040-fcc1916a4fdb None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:49:03.982076 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 575/1113] 10.222.0.46 () {56 vars in 1177 bytes} [Thu Jun 23 15:49:03 2022] GET /identity/v3/roles/d4d557a3ec4e434f8e83499c5fcc7136 => generated 223 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:49:03.988840 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d18492-c880-4bac-b903-4e787e69b411 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:49:04.084660 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:49:04.094291 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d18492-c880-4bac-b903-4e787e69b411 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:49:04.128271 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e989ccd-13c1-4581-b4af-94a2790fb237 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:49:04.129157 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e989ccd-13c1-4581-b4af-94a2790fb237 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:49:04.129438 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e989ccd-13c1-4581-b4af-94a2790fb237 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:49:04.129712 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e989ccd-13c1-4581-b4af-94a2790fb237 None admin] PATH_INFO: `/v3/users/e6f91d0a222944c4b510d2b8fed2892a` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:49:04.142844 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e989ccd-13c1-4581-b4af-94a2790fb237 None admin] RBAC: Authorizing `identity:get_user(user_id=e6f91d0a222944c4b510d2b8fed2892a)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:49:04.144332 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e989ccd-13c1-4581-b4af-94a2790fb237 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:49:04.146011 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 539/1114] 10.222.0.46 () {56 vars in 1177 bytes} [Thu Jun 23 15:49:03 2022] GET /identity/v3/users/e6f91d0a222944c4b510d2b8fed2892a => generated 286 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:49:04.152116 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19a2bfe-2657-47e3-a040-fcc1916a4fdb None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:49:04.218841 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:49:04.253489 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19a2bfe-2657-47e3-a040-fcc1916a4fdb None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:49:04.287383 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184ab181-5ea9-4ff6-bb35-3a3139ad6c56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:49:04.288030 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184ab181-5ea9-4ff6-bb35-3a3139ad6c56 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:49:04.288313 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184ab181-5ea9-4ff6-bb35-3a3139ad6c56 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:49:04.288591 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184ab181-5ea9-4ff6-bb35-3a3139ad6c56 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:49:04.356470 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184ab181-5ea9-4ff6-bb35-3a3139ad6c56 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:49:04.357838 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184ab181-5ea9-4ff6-bb35-3a3139ad6c56 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:49:04.424265 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:49:04.460850 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 576/1115] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Jun 23 15:49:04 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 23 15:49:04.466954 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e989ccd-13c1-4581-b4af-94a2790fb237 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:49:04.564907 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e989ccd-13c1-4581-b4af-94a2790fb237 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:49:04.599063 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4979dde8-e97c-4b8c-a798-5450045ac96d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:49:04.599939 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4979dde8-e97c-4b8c-a798-5450045ac96d None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:49:04.600215 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4979dde8-e97c-4b8c-a798-5450045ac96d None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:49:04.600490 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4979dde8-e97c-4b8c-a798-5450045ac96d None admin] PATH_INFO: `/v3/projects/68a2c61ef29c405e9509ad7cc3121def` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:49:04.601655 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4979dde8-e97c-4b8c-a798-5450045ac96d None admin] RBAC: Authorizing `identity:get_project(project_id=68a2c61ef29c405e9509ad7cc3121def)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:49:04.603113 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4979dde8-e97c-4b8c-a798-5450045ac96d None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:49:04.604804 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 540/1116] 10.222.0.46 () {56 vars in 1186 bytes} [Thu Jun 23 15:49:04 2022] GET /identity/v3/projects/68a2c61ef29c405e9509ad7cc3121def => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:49:04.610902 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184ab181-5ea9-4ff6-bb35-3a3139ad6c56 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:49:04.710571 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184ab181-5ea9-4ff6-bb35-3a3139ad6c56 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:49:04.745252 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ecb2e5-4cb7-4527-b8e5-6b40fa9b8a02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:49:04.745903 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ecb2e5-4cb7-4527-b8e5-6b40fa9b8a02 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:49:04.746174 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ecb2e5-4cb7-4527-b8e5-6b40fa9b8a02 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:49:04.746449 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ecb2e5-4cb7-4527-b8e5-6b40fa9b8a02 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:49:04.747304 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ecb2e5-4cb7-4527-b8e5-6b40fa9b8a02 None admin] RBAC: Adding query filter params (role.id=d4d557a3ec4e434f8e83499c5fcc7136, scope.project.id=68a2c61ef29c405e9509ad7cc3121def, user.id=e6f91d0a222944c4b510d2b8fed2892a) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:49:04.747619 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ecb2e5-4cb7-4527-b8e5-6b40fa9b8a02 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:49:04.749038 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ecb2e5-4cb7-4527-b8e5-6b40fa9b8a02 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:49:04.754453 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 577/1117] 10.222.0.46 () {56 vars in 1374 bytes} [Thu Jun 23 15:49:04 2022] GET /identity/v3/role_assignments?user.id=e6f91d0a222944c4b510d2b8fed2892a&scope.project.id=68a2c61ef29c405e9509ad7cc3121def&role.id=d4d557a3ec4e434f8e83499c5fcc7136 => generated 259 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:49:06.751733 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5963a10-006c-42e4-9da7-6743830c5898 None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:49:06.752277 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5963a10-006c-42e4-9da7-6743830c5898 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:49:06.752277 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5963a10-006c-42e4-9da7-6743830c5898 None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:49:06.753668 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 541/1118] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jun 23 15:49:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 15:49:06.758778 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a18c49-fd01-486a-9d8c-cafccf580ee4 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:49:06.759166 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a18c49-fd01-486a-9d8c-cafccf580ee4 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:49:06.759343 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a18c49-fd01-486a-9d8c-cafccf580ee4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:49:06.776301 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-89a18c49-fd01-486a-9d8c-cafccf580ee4 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:49:06.777546 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-89a18c49-fd01-486a-9d8c-cafccf580ee4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:49:06.802036 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:49:06.844739 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 578/1119] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jun 23 15:49:06 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 15:49:06.868463 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5963a10-006c-42e4-9da7-6743830c5898 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:49:06.889480 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:49:06.974542 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5963a10-006c-42e4-9da7-6743830c5898 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:49:07.008971 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db5a7c8-91b9-4f54-a6ab-487583df3dd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:49:07.009834 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db5a7c8-91b9-4f54-a6ab-487583df3dd0 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:49:07.010261 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db5a7c8-91b9-4f54-a6ab-487583df3dd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:49:07.010390 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db5a7c8-91b9-4f54-a6ab-487583df3dd0 None admin] PATH_INFO: `/v3/roles/d4d557a3ec4e434f8e83499c5fcc7136` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:49:07.011890 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db5a7c8-91b9-4f54-a6ab-487583df3dd0 None admin] RBAC: Authorizing `identity:get_role(role_id=d4d557a3ec4e434f8e83499c5fcc7136)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:49:07.013274 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db5a7c8-91b9-4f54-a6ab-487583df3dd0 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:49:07.014646 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 542/1120] 10.222.0.46 () {56 vars in 1177 bytes} [Thu Jun 23 15:49:06 2022] GET /identity/v3/roles/d4d557a3ec4e434f8e83499c5fcc7136 => generated 223 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:49:07.021244 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a18c49-fd01-486a-9d8c-cafccf580ee4 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:49:07.122460 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a18c49-fd01-486a-9d8c-cafccf580ee4 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:49:07.155974 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93735ac5-471b-409c-9b30-fa711caf633c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:49:07.156857 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93735ac5-471b-409c-9b30-fa711caf633c None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:49:07.157138 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93735ac5-471b-409c-9b30-fa711caf633c None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:49:07.157412 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93735ac5-471b-409c-9b30-fa711caf633c None admin] PATH_INFO: `/v3/users/e6f91d0a222944c4b510d2b8fed2892a` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:49:07.158599 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93735ac5-471b-409c-9b30-fa711caf633c None admin] RBAC: Authorizing `identity:get_user(user_id=e6f91d0a222944c4b510d2b8fed2892a)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:49:07.160069 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93735ac5-471b-409c-9b30-fa711caf633c None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:49:07.161698 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 579/1121] 10.222.0.46 () {56 vars in 1177 bytes} [Thu Jun 23 15:49:07 2022] GET /identity/v3/users/e6f91d0a222944c4b510d2b8fed2892a => generated 286 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:49:07.167804 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db5a7c8-91b9-4f54-a6ab-487583df3dd0 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:49:07.250662 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:49:07.266024 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db5a7c8-91b9-4f54-a6ab-487583df3dd0 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:49:07.299719 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef22c60c-c490-4c32-b291-704972a8bcfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:49:07.300640 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef22c60c-c490-4c32-b291-704972a8bcfe None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:49:07.300925 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef22c60c-c490-4c32-b291-704972a8bcfe None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:49:07.301202 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef22c60c-c490-4c32-b291-704972a8bcfe None admin] PATH_INFO: `/v3/projects/68a2c61ef29c405e9509ad7cc3121def` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:49:07.302452 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef22c60c-c490-4c32-b291-704972a8bcfe None admin] RBAC: Authorizing `identity:get_project(project_id=68a2c61ef29c405e9509ad7cc3121def)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:49:07.303927 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef22c60c-c490-4c32-b291-704972a8bcfe None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:49:07.305659 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 543/1122] 10.222.0.46 () {56 vars in 1186 bytes} [Thu Jun 23 15:49:07 2022] GET /identity/v3/projects/68a2c61ef29c405e9509ad7cc3121def => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:49:07.311607 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93735ac5-471b-409c-9b30-fa711caf633c None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:49:07.339967 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:49:07.413778 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93735ac5-471b-409c-9b30-fa711caf633c None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:49:07.446881 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5395e1-8759-4837-8d26-bbbdd522bc42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:49:07.447769 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5395e1-8759-4837-8d26-bbbdd522bc42 None admin] REQUEST_METHOD: `PUT` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:49:07.448052 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5395e1-8759-4837-8d26-bbbdd522bc42 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:49:07.448323 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5395e1-8759-4837-8d26-bbbdd522bc42 None admin] PATH_INFO: `/v3/projects/68a2c61ef29c405e9509ad7cc3121def/users/e6f91d0a222944c4b510d2b8fed2892a/roles/d4d557a3ec4e434f8e83499c5fcc7136` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:49:07.450507 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5395e1-8759-4837-8d26-bbbdd522bc42 None admin] RBAC: Authorizing `identity:create_grant(project_id=68a2c61ef29c405e9509ad7cc3121def, user_id=e6f91d0a222944c4b510d2b8fed2892a, role_id=d4d557a3ec4e434f8e83499c5fcc7136)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:49:07.451969 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5395e1-8759-4837-8d26-bbbdd522bc42 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:49:07.465057 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 580/1123] 10.222.0.46 () {58 vars in 1439 bytes} [Thu Jun 23 15:49:07 2022] PUT /identity/v3/projects/68a2c61ef29c405e9509ad7cc3121def/users/e6f91d0a222944c4b510d2b8fed2892a/roles/d4d557a3ec4e434f8e83499c5fcc7136 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 15:49:09.495757 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61891e1a-968e-4f73-8538-864019e0ca7a None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:49:09.496297 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61891e1a-968e-4f73-8538-864019e0ca7a None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:49:09.496297 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61891e1a-968e-4f73-8538-864019e0ca7a None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:49:09.497696 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 544/1124] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jun 23 15:49:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 15:49:09.504945 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c02065-8179-46d0-af9f-3e318edf909d None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:49:09.505433 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c02065-8179-46d0-af9f-3e318edf909d None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:49:09.505524 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c02065-8179-46d0-af9f-3e318edf909d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:49:09.522309 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-f5c02065-8179-46d0-af9f-3e318edf909d None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:49:09.523550 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-f5c02065-8179-46d0-af9f-3e318edf909d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:49:09.599344 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 581/1125] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jun 23 15:49:09 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 15:49:09.623448 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61891e1a-968e-4f73-8538-864019e0ca7a None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:49:09.725017 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61891e1a-968e-4f73-8538-864019e0ca7a None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:49:09.758607 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668daafe-fb20-454b-b7a7-06a5cbc02321 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:49:09.759454 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668daafe-fb20-454b-b7a7-06a5cbc02321 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:49:09.759739 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668daafe-fb20-454b-b7a7-06a5cbc02321 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:49:09.760016 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668daafe-fb20-454b-b7a7-06a5cbc02321 None admin] PATH_INFO: `/v3/roles/d4d557a3ec4e434f8e83499c5fcc7136` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:49:09.761491 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668daafe-fb20-454b-b7a7-06a5cbc02321 None admin] RBAC: Authorizing `identity:get_role(role_id=d4d557a3ec4e434f8e83499c5fcc7136)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:49:09.762905 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668daafe-fb20-454b-b7a7-06a5cbc02321 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:49:09.764299 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 545/1126] 10.222.0.46 () {56 vars in 1177 bytes} [Thu Jun 23 15:49:09 2022] GET /identity/v3/roles/d4d557a3ec4e434f8e83499c5fcc7136 => generated 223 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:49:09.770245 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c02065-8179-46d0-af9f-3e318edf909d None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:49:09.788989 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:49:09.888109 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c02065-8179-46d0-af9f-3e318edf909d None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:49:09.922574 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd1e345-0c7d-4807-9fe3-970e45304d3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:49:09.923471 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd1e345-0c7d-4807-9fe3-970e45304d3e None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:49:09.923752 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd1e345-0c7d-4807-9fe3-970e45304d3e None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:49:09.924024 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd1e345-0c7d-4807-9fe3-970e45304d3e None admin] PATH_INFO: `/v3/users/e6f91d0a222944c4b510d2b8fed2892a` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:49:09.925832 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd1e345-0c7d-4807-9fe3-970e45304d3e None admin] RBAC: Authorizing `identity:get_user(user_id=e6f91d0a222944c4b510d2b8fed2892a)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:49:09.927303 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd1e345-0c7d-4807-9fe3-970e45304d3e None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:49:09.928939 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 582/1127] 10.222.0.46 () {56 vars in 1177 bytes} [Thu Jun 23 15:49:09 2022] GET /identity/v3/users/e6f91d0a222944c4b510d2b8fed2892a => generated 286 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:49:09.934901 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668daafe-fb20-454b-b7a7-06a5cbc02321 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:49:09.958447 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:49:10.038006 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668daafe-fb20-454b-b7a7-06a5cbc02321 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:49:10.070295 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfafa16-7ec4-4c4d-89d4-9f751a76fd0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:49:10.070984 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfafa16-7ec4-4c4d-89d4-9f751a76fd0c None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:49:10.071263 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfafa16-7ec4-4c4d-89d4-9f751a76fd0c None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:49:10.071538 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfafa16-7ec4-4c4d-89d4-9f751a76fd0c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:49:10.136769 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfafa16-7ec4-4c4d-89d4-9f751a76fd0c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:49:10.138206 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfafa16-7ec4-4c4d-89d4-9f751a76fd0c None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:49:10.161476 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:49:10.237305 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 546/1128] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Jun 23 15:49:09 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 23 15:49:10.243200 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd1e345-0c7d-4807-9fe3-970e45304d3e None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:49:10.326938 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:49:10.342244 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd1e345-0c7d-4807-9fe3-970e45304d3e None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:49:10.376271 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0392f1-5464-43ea-84a0-4c3c965b366a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:49:10.377136 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0392f1-5464-43ea-84a0-4c3c965b366a None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:49:10.377953 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0392f1-5464-43ea-84a0-4c3c965b366a None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:49:10.378231 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0392f1-5464-43ea-84a0-4c3c965b366a None admin] PATH_INFO: `/v3/projects/68a2c61ef29c405e9509ad7cc3121def` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:49:10.379429 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0392f1-5464-43ea-84a0-4c3c965b366a None admin] RBAC: Authorizing `identity:get_project(project_id=68a2c61ef29c405e9509ad7cc3121def)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:49:10.380866 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0392f1-5464-43ea-84a0-4c3c965b366a None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:49:10.382565 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 583/1129] 10.222.0.46 () {56 vars in 1186 bytes} [Thu Jun 23 15:49:10 2022] GET /identity/v3/projects/68a2c61ef29c405e9509ad7cc3121def => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:49:10.388694 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfafa16-7ec4-4c4d-89d4-9f751a76fd0c None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:49:10.486705 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfafa16-7ec4-4c4d-89d4-9f751a76fd0c None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:49:10.510033 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:49:10.522383 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913e64e6-fae1-4ca2-a198-a72071f73d9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:49:10.523086 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913e64e6-fae1-4ca2-a198-a72071f73d9d None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:49:10.523372 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913e64e6-fae1-4ca2-a198-a72071f73d9d None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:49:10.523906 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913e64e6-fae1-4ca2-a198-a72071f73d9d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:49:10.524734 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913e64e6-fae1-4ca2-a198-a72071f73d9d None admin] RBAC: Adding query filter params (role.id=d4d557a3ec4e434f8e83499c5fcc7136, scope.project.id=68a2c61ef29c405e9509ad7cc3121def, user.id=e6f91d0a222944c4b510d2b8fed2892a) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:49:10.525048 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913e64e6-fae1-4ca2-a198-a72071f73d9d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:49:10.526493 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913e64e6-fae1-4ca2-a198-a72071f73d9d None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:49:10.532369 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 547/1130] 10.222.0.46 () {56 vars in 1374 bytes} [Thu Jun 23 15:49:10 2022] GET /identity/v3/role_assignments?user.id=e6f91d0a222944c4b510d2b8fed2892a&scope.project.id=68a2c61ef29c405e9509ad7cc3121def&role.id=d4d557a3ec4e434f8e83499c5fcc7136 => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:49:12.600557 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccbf12b-6e43-4516-b2f9-4b44f37de363 None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:49:12.601041 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccbf12b-6e43-4516-b2f9-4b44f37de363 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:49:12.601132 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccbf12b-6e43-4516-b2f9-4b44f37de363 None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:49:12.602476 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 584/1131] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jun 23 15:49:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 15:49:12.607738 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e72da8-e899-478b-b4da-4e06cbb9a5a1 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:49:12.608174 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e72da8-e899-478b-b4da-4e06cbb9a5a1 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:49:12.608290 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e72da8-e899-478b-b4da-4e06cbb9a5a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:49:12.625145 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-a7e72da8-e899-478b-b4da-4e06cbb9a5a1 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:49:12.626291 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-a7e72da8-e899-478b-b4da-4e06cbb9a5a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:49:12.692000 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 548/1132] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jun 23 15:49:12 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 15:49:12.715673 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccbf12b-6e43-4516-b2f9-4b44f37de363 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:49:12.819490 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccbf12b-6e43-4516-b2f9-4b44f37de363 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:49:12.853518 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af830420-9fdf-4423-b65c-5dc61490ef53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:49:12.854312 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af830420-9fdf-4423-b65c-5dc61490ef53 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:49:12.854599 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af830420-9fdf-4423-b65c-5dc61490ef53 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:49:12.854907 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af830420-9fdf-4423-b65c-5dc61490ef53 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:49:12.856087 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af830420-9fdf-4423-b65c-5dc61490ef53 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:49:12.857532 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af830420-9fdf-4423-b65c-5dc61490ef53 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:49:12.859201 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 585/1133] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Jun 23 15:49:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:49:12.865198 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e72da8-e899-478b-b4da-4e06cbb9a5a1 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:49:12.961046 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e72da8-e899-478b-b4da-4e06cbb9a5a1 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:49:12.977442 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:49:12.996853 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa7ee60-dea5-4164-9027-1a60bed716ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:49:12.997519 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa7ee60-dea5-4164-9027-1a60bed716ac None admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:49:12.997802 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa7ee60-dea5-4164-9027-1a60bed716ac None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:49:12.998077 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa7ee60-dea5-4164-9027-1a60bed716ac None admin] PATH_INFO: `/v3/projects` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:49:12.998822 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa7ee60-dea5-4164-9027-1a60bed716ac None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:49:13.000283 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa7ee60-dea5-4164-9027-1a60bed716ac None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:49:13.025502 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 549/1134] 10.222.0.46 () {60 vars in 1141 bytes} [Thu Jun 23 15:49:12 2022] POST /identity/v3/projects => generated 311 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 15:49:15.106234 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2546c024-dba7-485e-b2fd-24d3d9415ee9 None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:49:15.106812 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2546c024-dba7-485e-b2fd-24d3d9415ee9 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:49:15.106812 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2546c024-dba7-485e-b2fd-24d3d9415ee9 None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:49:15.108219 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 586/1135] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jun 23 15:49:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 15:49:15.113516 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef39fc28-43d7-4ade-b3bb-9c1cb5c70daa None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:49:15.113971 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef39fc28-43d7-4ade-b3bb-9c1cb5c70daa None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:49:15.114072 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef39fc28-43d7-4ade-b3bb-9c1cb5c70daa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:49:15.132339 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-ef39fc28-43d7-4ade-b3bb-9c1cb5c70daa None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:49:15.133544 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-ef39fc28-43d7-4ade-b3bb-9c1cb5c70daa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:49:15.204954 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 550/1136] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jun 23 15:49:15 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 15:49:15.230546 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2546c024-dba7-485e-b2fd-24d3d9415ee9 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:49:15.256472 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:49:15.341168 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2546c024-dba7-485e-b2fd-24d3d9415ee9 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:49:15.375229 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ef349e-92d3-46f9-aade-2ef812551734 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:49:15.375944 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ef349e-92d3-46f9-aade-2ef812551734 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:49:15.376228 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ef349e-92d3-46f9-aade-2ef812551734 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:49:15.376502 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ef349e-92d3-46f9-aade-2ef812551734 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:49:15.378336 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ef349e-92d3-46f9-aade-2ef812551734 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:49:15.379815 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ef349e-92d3-46f9-aade-2ef812551734 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:49:15.381438 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 587/1137] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Jun 23 15:49:15 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:49:15.387277 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef39fc28-43d7-4ade-b3bb-9c1cb5c70daa None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:49:15.481727 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:49:15.485383 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef39fc28-43d7-4ade-b3bb-9c1cb5c70daa None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:49:15.521116 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e902eb-a096-4127-877e-28dd91621f7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:49:15.521795 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e902eb-a096-4127-877e-28dd91621f7c None admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:49:15.522084 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e902eb-a096-4127-877e-28dd91621f7c None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:49:15.522361 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e902eb-a096-4127-877e-28dd91621f7c None admin] PATH_INFO: `/v3/users` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:49:15.523100 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e902eb-a096-4127-877e-28dd91621f7c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:49:15.524806 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e902eb-a096-4127-877e-28dd91621f7c None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:49:15.544601 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 551/1138] 10.222.0.46 () {60 vars in 1132 bytes} [Thu Jun 23 15:49:15 2022] POST /identity/v3/users => generated 286 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 15:49:17.643279 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cb3882-c3e0-4849-a203-c42a3b247d4a None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:49:17.643805 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cb3882-c3e0-4849-a203-c42a3b247d4a None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:49:17.644262 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cb3882-c3e0-4849-a203-c42a3b247d4a None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:49:17.646813 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 588/1139] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jun 23 15:49:17 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 15:49:17.652623 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5d8ae6-bafc-4b42-9f33-7fe88c3e3354 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:49:17.653729 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5d8ae6-bafc-4b42-9f33-7fe88c3e3354 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:49:17.653729 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5d8ae6-bafc-4b42-9f33-7fe88c3e3354 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:49:17.670106 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-0e5d8ae6-bafc-4b42-9f33-7fe88c3e3354 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:49:17.671315 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-0e5d8ae6-bafc-4b42-9f33-7fe88c3e3354 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:49:17.737104 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 552/1140] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jun 23 15:49:17 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 15:49:17.760924 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cb3882-c3e0-4849-a203-c42a3b247d4a None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:49:17.858546 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:49:17.870954 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cb3882-c3e0-4849-a203-c42a3b247d4a None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:49:17.909148 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb96588-f58b-468d-8daf-99ec8028e295 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:49:17.909961 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb96588-f58b-468d-8daf-99ec8028e295 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:49:17.910352 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb96588-f58b-468d-8daf-99ec8028e295 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:49:17.910633 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb96588-f58b-468d-8daf-99ec8028e295 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:49:17.922185 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb96588-f58b-468d-8daf-99ec8028e295 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 15:49:17.929992 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-deb96588-f58b-468d-8daf-99ec8028e295 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 15:49:17.929992 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:49:17.929992 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:49:17.929992 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:49:17.929992 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:49:17.929992 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:49:17.929992 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:49:17.929992 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:49:17.929992 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:49:17.929992 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:49:17.929992 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:49:17.929992 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:49:17.929992 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 15:49:17.929992 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 15:49:17.929992 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 15:49:17.929992 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 15:49:17.931278 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 15:49:17.931278 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 15:49:17.931278 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 15:49:17.931278 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 15:49:17.931278 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:49:17.931278 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:49:17.931278 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:49:17.931278 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:49:17.931278 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:49:17.931278 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:49:17.931278 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:49:17.931278 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Jun 23 15:49:17.931278 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:49:17.931278 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:49:17.931278 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:49:17.931278 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:49:17.932560 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:49:17.932560 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:49:17.932560 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:49:17.932560 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:49:17.932560 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 15:49:17.932560 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 15:49:17.932560 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 15:49:17.932560 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 15:49:17.932560 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 15:49:17.932560 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 15:49:17.932560 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 15:49:17.932560 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Jun 23 15:49:17.932560 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 589/1141] 10.222.0.46 () {56 vars in 1096 bytes} [Thu Jun 23 15:49:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:49:17.937546 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5d8ae6-bafc-4b42-9f33-7fe88c3e3354 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:49:18.025888 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:49:18.041694 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5d8ae6-bafc-4b42-9f33-7fe88c3e3354 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:49:18.075998 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be48419-4b55-4ca6-9f1a-d041bf598bba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:49:18.076671 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be48419-4b55-4ca6-9f1a-d041bf598bba None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:49:18.076949 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be48419-4b55-4ca6-9f1a-d041bf598bba None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:49:18.077223 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be48419-4b55-4ca6-9f1a-d041bf598bba None admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:49:18.077964 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be48419-4b55-4ca6-9f1a-d041bf598bba None admin] RBAC: Adding query filter params (name=admin) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:49:18.078264 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be48419-4b55-4ca6-9f1a-d041bf598bba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:49:18.079676 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be48419-4b55-4ca6-9f1a-d041bf598bba None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:49:18.088384 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 553/1142] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Jun 23 15:49:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:49:18.094181 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb96588-f58b-468d-8daf-99ec8028e295 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:49:18.194127 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb96588-f58b-468d-8daf-99ec8028e295 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:49:18.227205 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4323a4-7e33-4b08-b3d3-76625022fe60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:49:18.228091 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4323a4-7e33-4b08-b3d3-76625022fe60 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:49:18.228373 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4323a4-7e33-4b08-b3d3-76625022fe60 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:49:18.228654 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4323a4-7e33-4b08-b3d3-76625022fe60 None admin] PATH_INFO: `/v3/users/9a9c9adda5cf4de1ae4b50bef51f87cf` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:49:18.243060 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4323a4-7e33-4b08-b3d3-76625022fe60 None admin] RBAC: Authorizing `identity:get_user(user_id=9a9c9adda5cf4de1ae4b50bef51f87cf)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:49:18.244524 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4323a4-7e33-4b08-b3d3-76625022fe60 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:49:18.246204 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 590/1143] 10.222.0.46 () {56 vars in 1177 bytes} [Thu Jun 23 15:49:18 2022] GET /identity/v3/users/9a9c9adda5cf4de1ae4b50bef51f87cf => generated 286 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:49:18.252739 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be48419-4b55-4ca6-9f1a-d041bf598bba None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:49:18.358658 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be48419-4b55-4ca6-9f1a-d041bf598bba None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:49:18.395705 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063fb158-9c93-413a-b847-9e0609aa3527 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:49:18.396363 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063fb158-9c93-413a-b847-9e0609aa3527 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:49:18.396649 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063fb158-9c93-413a-b847-9e0609aa3527 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:49:18.396925 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063fb158-9c93-413a-b847-9e0609aa3527 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:49:18.412415 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:49:18.464643 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063fb158-9c93-413a-b847-9e0609aa3527 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:49:18.466018 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063fb158-9c93-413a-b847-9e0609aa3527 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:49:18.572450 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 554/1144] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Jun 23 15:49:18 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 23 15:49:18.578283 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4323a4-7e33-4b08-b3d3-76625022fe60 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:49:18.589188 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:49:18.676273 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4323a4-7e33-4b08-b3d3-76625022fe60 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:49:18.709840 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96defb46-22da-41fd-842b-3fb4558c8fef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:49:18.710729 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96defb46-22da-41fd-842b-3fb4558c8fef None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:49:18.711056 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96defb46-22da-41fd-842b-3fb4558c8fef None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:49:18.711263 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96defb46-22da-41fd-842b-3fb4558c8fef None admin] PATH_INFO: `/v3/projects/634af2cdd9c04aa4bf1a73f4ebc3dd7d` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:49:18.712420 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96defb46-22da-41fd-842b-3fb4558c8fef None admin] RBAC: Authorizing `identity:get_project(project_id=634af2cdd9c04aa4bf1a73f4ebc3dd7d)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:49:18.713925 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96defb46-22da-41fd-842b-3fb4558c8fef None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:49:18.715682 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 591/1145] 10.222.0.46 () {56 vars in 1186 bytes} [Thu Jun 23 15:49:18 2022] GET /identity/v3/projects/634af2cdd9c04aa4bf1a73f4ebc3dd7d => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:49:18.722561 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063fb158-9c93-413a-b847-9e0609aa3527 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:49:18.767297 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:49:18.821804 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063fb158-9c93-413a-b847-9e0609aa3527 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:49:18.855130 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b8908a-85dc-42c0-954d-26f35ce4c312 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:49:18.855796 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b8908a-85dc-42c0-954d-26f35ce4c312 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:49:18.856085 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b8908a-85dc-42c0-954d-26f35ce4c312 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:49:18.856366 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b8908a-85dc-42c0-954d-26f35ce4c312 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:49:18.857198 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b8908a-85dc-42c0-954d-26f35ce4c312 None admin] RBAC: Adding query filter params (role.id=812d3c567c52469cb2b35709fbdc857c, scope.project.id=634af2cdd9c04aa4bf1a73f4ebc3dd7d, user.id=9a9c9adda5cf4de1ae4b50bef51f87cf) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:49:18.857523 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b8908a-85dc-42c0-954d-26f35ce4c312 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:49:18.858989 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b8908a-85dc-42c0-954d-26f35ce4c312 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:49:18.865024 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 555/1146] 10.222.0.46 () {56 vars in 1374 bytes} [Thu Jun 23 15:49:18 2022] GET /identity/v3/role_assignments?user.id=9a9c9adda5cf4de1ae4b50bef51f87cf&scope.project.id=634af2cdd9c04aa4bf1a73f4ebc3dd7d&role.id=812d3c567c52469cb2b35709fbdc857c => generated 259 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:49:20.880273 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84689fa8-9636-419c-87d4-f89561e04ea1 None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:49:20.880837 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84689fa8-9636-419c-87d4-f89561e04ea1 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:49:20.880837 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84689fa8-9636-419c-87d4-f89561e04ea1 None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:49:20.882174 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 592/1147] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jun 23 15:49:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 15:49:20.887298 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfaf9381-a313-4c32-879e-9966b0ebd61c None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:49:20.887681 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfaf9381-a313-4c32-879e-9966b0ebd61c None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:49:20.887851 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfaf9381-a313-4c32-879e-9966b0ebd61c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:49:20.904985 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-bfaf9381-a313-4c32-879e-9966b0ebd61c None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:49:20.906160 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-bfaf9381-a313-4c32-879e-9966b0ebd61c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:49:20.969384 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 556/1148] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jun 23 15:49:20 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 15:49:20.991807 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84689fa8-9636-419c-87d4-f89561e04ea1 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:49:21.068443 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:49:21.091884 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84689fa8-9636-419c-87d4-f89561e04ea1 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:49:21.124205 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30cd82c-70a2-4745-9120-41e9353f15f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:49:21.124986 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30cd82c-70a2-4745-9120-41e9353f15f2 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:49:21.125264 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30cd82c-70a2-4745-9120-41e9353f15f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:49:21.125622 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30cd82c-70a2-4745-9120-41e9353f15f2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:49:21.136418 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30cd82c-70a2-4745-9120-41e9353f15f2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 15:49:21.145087 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-c30cd82c-70a2-4745-9120-41e9353f15f2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 15:49:21.145087 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:49:21.145087 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:49:21.145087 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:49:21.145087 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:49:21.145087 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:49:21.145087 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:49:21.145087 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:49:21.145087 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:49:21.145087 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:49:21.145087 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:49:21.145087 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:49:21.145087 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 15:49:21.145087 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 15:49:21.145087 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 15:49:21.145087 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 15:49:21.146190 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 15:49:21.146190 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 15:49:21.146190 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 15:49:21.146190 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 15:49:21.146190 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:49:21.146190 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:49:21.146190 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:49:21.146190 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:49:21.146190 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:49:21.146190 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:49:21.146190 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:49:21.146190 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Jun 23 15:49:21.146190 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:49:21.146190 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:49:21.146190 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:49:21.146190 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:49:21.147494 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:49:21.147494 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:49:21.147494 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:49:21.147494 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:49:21.147494 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 15:49:21.147494 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 15:49:21.147494 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 15:49:21.147494 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 15:49:21.147494 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 15:49:21.147494 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 15:49:21.147494 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 15:49:21.147494 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Jun 23 15:49:21.147494 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 593/1149] 10.222.0.46 () {56 vars in 1096 bytes} [Thu Jun 23 15:49:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:49:21.152679 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfaf9381-a313-4c32-879e-9966b0ebd61c None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:49:21.192689 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:49:21.250196 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfaf9381-a313-4c32-879e-9966b0ebd61c None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:49:21.283436 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57abde28-e997-4c4a-964c-250676a4fd44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:49:21.284387 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57abde28-e997-4c4a-964c-250676a4fd44 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:49:21.284705 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57abde28-e997-4c4a-964c-250676a4fd44 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:49:21.284990 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57abde28-e997-4c4a-964c-250676a4fd44 None admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:49:21.285744 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57abde28-e997-4c4a-964c-250676a4fd44 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:49:21.286057 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57abde28-e997-4c4a-964c-250676a4fd44 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:49:21.287457 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57abde28-e997-4c4a-964c-250676a4fd44 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:49:21.295377 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 557/1150] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Jun 23 15:49:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:49:21.301712 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30cd82c-70a2-4745-9120-41e9353f15f2 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:49:21.399519 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30cd82c-70a2-4745-9120-41e9353f15f2 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:49:21.432012 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095a858e-75e8-4fc7-8d99-17ccaf721544 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:49:21.432878 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095a858e-75e8-4fc7-8d99-17ccaf721544 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:49:21.433160 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095a858e-75e8-4fc7-8d99-17ccaf721544 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:49:21.433437 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095a858e-75e8-4fc7-8d99-17ccaf721544 None admin] PATH_INFO: `/v3/users/9a9c9adda5cf4de1ae4b50bef51f87cf` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:49:21.434617 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095a858e-75e8-4fc7-8d99-17ccaf721544 None admin] RBAC: Authorizing `identity:get_user(user_id=9a9c9adda5cf4de1ae4b50bef51f87cf)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:49:21.436092 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095a858e-75e8-4fc7-8d99-17ccaf721544 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:49:21.437713 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 594/1151] 10.222.0.46 () {56 vars in 1177 bytes} [Thu Jun 23 15:49:21 2022] GET /identity/v3/users/9a9c9adda5cf4de1ae4b50bef51f87cf => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:49:21.443495 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57abde28-e997-4c4a-964c-250676a4fd44 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:49:21.540746 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57abde28-e997-4c4a-964c-250676a4fd44 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:49:21.542270 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:49:21.576424 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c80cf2-8483-47dd-96e0-89a42a9db58e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:49:21.577195 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c80cf2-8483-47dd-96e0-89a42a9db58e None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:49:21.577474 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c80cf2-8483-47dd-96e0-89a42a9db58e None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:49:21.577744 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c80cf2-8483-47dd-96e0-89a42a9db58e None admin] PATH_INFO: `/v3/projects/634af2cdd9c04aa4bf1a73f4ebc3dd7d` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:49:21.578942 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c80cf2-8483-47dd-96e0-89a42a9db58e None admin] RBAC: Authorizing `identity:get_project(project_id=634af2cdd9c04aa4bf1a73f4ebc3dd7d)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:49:21.580910 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c80cf2-8483-47dd-96e0-89a42a9db58e None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:49:21.582591 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 558/1152] 10.222.0.46 () {56 vars in 1186 bytes} [Thu Jun 23 15:49:21 2022] GET /identity/v3/projects/634af2cdd9c04aa4bf1a73f4ebc3dd7d => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:49:21.588663 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095a858e-75e8-4fc7-8d99-17ccaf721544 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:49:21.592477 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:49:21.696980 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095a858e-75e8-4fc7-8d99-17ccaf721544 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:49:21.729351 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07cf665-8138-4f0b-9697-3d106fede37f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:49:21.730233 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07cf665-8138-4f0b-9697-3d106fede37f None admin] REQUEST_METHOD: `PUT` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:49:21.730515 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07cf665-8138-4f0b-9697-3d106fede37f None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:49:21.730815 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07cf665-8138-4f0b-9697-3d106fede37f None admin] PATH_INFO: `/v3/projects/634af2cdd9c04aa4bf1a73f4ebc3dd7d/users/9a9c9adda5cf4de1ae4b50bef51f87cf/roles/812d3c567c52469cb2b35709fbdc857c` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:49:21.732741 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07cf665-8138-4f0b-9697-3d106fede37f None admin] RBAC: Authorizing `identity:create_grant(project_id=634af2cdd9c04aa4bf1a73f4ebc3dd7d, user_id=9a9c9adda5cf4de1ae4b50bef51f87cf, role_id=812d3c567c52469cb2b35709fbdc857c)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:49:21.734170 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07cf665-8138-4f0b-9697-3d106fede37f None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:49:21.747865 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 595/1153] 10.222.0.46 () {58 vars in 1439 bytes} [Thu Jun 23 15:49:21 2022] PUT /identity/v3/projects/634af2cdd9c04aa4bf1a73f4ebc3dd7d/users/9a9c9adda5cf4de1ae4b50bef51f87cf/roles/812d3c567c52469cb2b35709fbdc857c => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 15:49:23.764618 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc9cb95-ea3f-45ba-b896-b9b3ed36e410 None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:49:23.765138 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc9cb95-ea3f-45ba-b896-b9b3ed36e410 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:49:23.765229 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc9cb95-ea3f-45ba-b896-b9b3ed36e410 None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:49:23.766532 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 559/1154] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jun 23 15:49:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 15:49:23.771935 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6581d045-07f8-424f-afdf-a0143c13abbd None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:49:23.772370 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6581d045-07f8-424f-afdf-a0143c13abbd None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:49:23.772496 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6581d045-07f8-424f-afdf-a0143c13abbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:49:23.789143 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-6581d045-07f8-424f-afdf-a0143c13abbd None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:49:23.790326 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-6581d045-07f8-424f-afdf-a0143c13abbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:49:23.847801 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:49:23.857216 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 596/1155] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jun 23 15:49:23 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 15:49:23.882294 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc9cb95-ea3f-45ba-b896-b9b3ed36e410 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:49:23.980825 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc9cb95-ea3f-45ba-b896-b9b3ed36e410 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:49:24.013318 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c34c6f-9c67-49e4-8034-c6c7bb8edcf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:49:24.014111 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c34c6f-9c67-49e4-8034-c6c7bb8edcf0 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:49:24.014400 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c34c6f-9c67-49e4-8034-c6c7bb8edcf0 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:49:24.014680 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c34c6f-9c67-49e4-8034-c6c7bb8edcf0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:49:24.025360 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c34c6f-9c67-49e4-8034-c6c7bb8edcf0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 15:49:24.033609 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-53c34c6f-9c67-49e4-8034-c6c7bb8edcf0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 15:49:24.033609 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:49:24.033609 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:49:24.033609 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:49:24.033609 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:49:24.033609 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:49:24.033609 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:49:24.033609 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:49:24.033609 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:49:24.033609 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:49:24.033609 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:49:24.033609 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:49:24.033609 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 15:49:24.033609 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 15:49:24.033609 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 15:49:24.033609 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 15:49:24.034722 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 15:49:24.034722 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 15:49:24.034722 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 15:49:24.034722 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 15:49:24.034722 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:49:24.034722 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:49:24.034722 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:49:24.034722 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:49:24.034722 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:49:24.034722 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:49:24.034722 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:49:24.034722 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Jun 23 15:49:24.034722 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:49:24.034722 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:49:24.034722 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:49:24.034722 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:49:24.035750 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:49:24.035750 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:49:24.035750 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:49:24.035750 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:49:24.035750 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 15:49:24.035750 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 15:49:24.035750 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 15:49:24.035750 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 15:49:24.035750 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 15:49:24.035750 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 15:49:24.035750 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 15:49:24.035750 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Jun 23 15:49:24.035750 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 560/1156] 10.222.0.46 () {56 vars in 1096 bytes} [Thu Jun 23 15:49:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:49:24.040989 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6581d045-07f8-424f-afdf-a0143c13abbd None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:49:24.135852 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6581d045-07f8-424f-afdf-a0143c13abbd None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:49:24.169171 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33effccd-4cda-4773-bb0c-b88beb6ea28a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:49:24.169828 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33effccd-4cda-4773-bb0c-b88beb6ea28a None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:49:24.170106 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33effccd-4cda-4773-bb0c-b88beb6ea28a None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:49:24.170375 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33effccd-4cda-4773-bb0c-b88beb6ea28a None admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:49:24.171135 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33effccd-4cda-4773-bb0c-b88beb6ea28a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:49:24.171439 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33effccd-4cda-4773-bb0c-b88beb6ea28a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:49:24.172799 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33effccd-4cda-4773-bb0c-b88beb6ea28a None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:49:24.181249 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 597/1157] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Jun 23 15:49:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:49:24.187030 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c34c6f-9c67-49e4-8034-c6c7bb8edcf0 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:49:24.216920 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:49:24.286282 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c34c6f-9c67-49e4-8034-c6c7bb8edcf0 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:49:24.318806 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9853296e-11d6-420c-a074-17a40e97541b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:49:24.319679 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9853296e-11d6-420c-a074-17a40e97541b None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:49:24.319967 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9853296e-11d6-420c-a074-17a40e97541b None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:49:24.320245 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9853296e-11d6-420c-a074-17a40e97541b None admin] PATH_INFO: `/v3/users/9a9c9adda5cf4de1ae4b50bef51f87cf` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:49:24.321429 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9853296e-11d6-420c-a074-17a40e97541b None admin] RBAC: Authorizing `identity:get_user(user_id=9a9c9adda5cf4de1ae4b50bef51f87cf)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:49:24.322891 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9853296e-11d6-420c-a074-17a40e97541b None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:49:24.324513 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 561/1158] 10.222.0.46 () {56 vars in 1177 bytes} [Thu Jun 23 15:49:24 2022] GET /identity/v3/users/9a9c9adda5cf4de1ae4b50bef51f87cf => generated 286 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:49:24.330868 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33effccd-4cda-4773-bb0c-b88beb6ea28a None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:49:24.385425 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:49:24.429355 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33effccd-4cda-4773-bb0c-b88beb6ea28a None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:49:24.461547 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70e479f-d93c-494f-a6df-3d0aedeb2fe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:49:24.462192 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70e479f-d93c-494f-a6df-3d0aedeb2fe4 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:49:24.462478 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70e479f-d93c-494f-a6df-3d0aedeb2fe4 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:49:24.462777 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70e479f-d93c-494f-a6df-3d0aedeb2fe4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:49:24.530629 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70e479f-d93c-494f-a6df-3d0aedeb2fe4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:49:24.532114 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70e479f-d93c-494f-a6df-3d0aedeb2fe4 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:49:24.587177 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:49:24.631802 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 598/1159] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Jun 23 15:49:24 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 23 15:49:24.637630 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9853296e-11d6-420c-a074-17a40e97541b None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:49:24.731028 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:49:24.735278 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9853296e-11d6-420c-a074-17a40e97541b None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:49:24.768775 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82f752e-5e06-4f2c-ac6a-a8ea2a5e21f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:49:24.769636 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82f752e-5e06-4f2c-ac6a-a8ea2a5e21f7 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:49:24.769915 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82f752e-5e06-4f2c-ac6a-a8ea2a5e21f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:49:24.770186 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82f752e-5e06-4f2c-ac6a-a8ea2a5e21f7 None admin] PATH_INFO: `/v3/projects/634af2cdd9c04aa4bf1a73f4ebc3dd7d` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:49:24.771373 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82f752e-5e06-4f2c-ac6a-a8ea2a5e21f7 None admin] RBAC: Authorizing `identity:get_project(project_id=634af2cdd9c04aa4bf1a73f4ebc3dd7d)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:49:24.772819 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82f752e-5e06-4f2c-ac6a-a8ea2a5e21f7 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:49:24.774783 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 562/1160] 10.222.0.46 () {56 vars in 1186 bytes} [Thu Jun 23 15:49:24 2022] GET /identity/v3/projects/634af2cdd9c04aa4bf1a73f4ebc3dd7d => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:49:24.780864 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70e479f-d93c-494f-a6df-3d0aedeb2fe4 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:49:24.878438 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70e479f-d93c-494f-a6df-3d0aedeb2fe4 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:49:24.914745 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc0d1f3-4631-4b56-9915-aa4b700f5ba1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:49:24.915401 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc0d1f3-4631-4b56-9915-aa4b700f5ba1 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:49:24.915682 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc0d1f3-4631-4b56-9915-aa4b700f5ba1 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:49:24.915957 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc0d1f3-4631-4b56-9915-aa4b700f5ba1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:49:24.916780 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc0d1f3-4631-4b56-9915-aa4b700f5ba1 None admin] RBAC: Adding query filter params (role.id=812d3c567c52469cb2b35709fbdc857c, scope.project.id=634af2cdd9c04aa4bf1a73f4ebc3dd7d, user.id=9a9c9adda5cf4de1ae4b50bef51f87cf) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:49:24.917088 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc0d1f3-4631-4b56-9915-aa4b700f5ba1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:49:24.918510 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc0d1f3-4631-4b56-9915-aa4b700f5ba1 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:49:24.924262 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 599/1161] 10.222.0.46 () {56 vars in 1374 bytes} [Thu Jun 23 15:49:24 2022] GET /identity/v3/role_assignments?user.id=9a9c9adda5cf4de1ae4b50bef51f87cf&scope.project.id=634af2cdd9c04aa4bf1a73f4ebc3dd7d&role.id=812d3c567c52469cb2b35709fbdc857c => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:49:26.953068 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0f5378-7171-4502-a9b6-04a680a4653b None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:49:26.953595 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0f5378-7171-4502-a9b6-04a680a4653b None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:49:26.953595 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0f5378-7171-4502-a9b6-04a680a4653b None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:49:26.955609 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 563/1162] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jun 23 15:49:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 15:49:26.960647 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1939848b-1f65-4219-9c75-c1189f5ce2ee None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:49:26.961039 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1939848b-1f65-4219-9c75-c1189f5ce2ee None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:49:26.961206 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1939848b-1f65-4219-9c75-c1189f5ce2ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:49:26.977483 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-1939848b-1f65-4219-9c75-c1189f5ce2ee None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:49:26.978646 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-1939848b-1f65-4219-9c75-c1189f5ce2ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:49:26.992329 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:49:27.045727 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 600/1163] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jun 23 15:49:26 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 15:49:27.068165 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0f5378-7171-4502-a9b6-04a680a4653b None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:49:27.166189 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0f5378-7171-4502-a9b6-04a680a4653b None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:49:27.198288 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c688d460-a2f0-40c5-b185-c20d087b494f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:49:27.198953 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c688d460-a2f0-40c5-b185-c20d087b494f None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:49:27.199237 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c688d460-a2f0-40c5-b185-c20d087b494f None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:49:27.199510 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c688d460-a2f0-40c5-b185-c20d087b494f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:49:27.232172 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:49:27.266849 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c688d460-a2f0-40c5-b185-c20d087b494f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:49:27.268308 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c688d460-a2f0-40c5-b185-c20d087b494f None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:49:27.362102 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 564/1164] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Jun 23 15:49:27 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 23 15:49:27.368145 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1939848b-1f65-4219-9c75-c1189f5ce2ee None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:49:27.463880 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1939848b-1f65-4219-9c75-c1189f5ce2ee None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:49:27.496948 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0380b06-68c8-43ed-b27a-05c86be1ebff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:49:27.497695 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0380b06-68c8-43ed-b27a-05c86be1ebff None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:49:27.497979 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0380b06-68c8-43ed-b27a-05c86be1ebff None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:49:27.498261 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0380b06-68c8-43ed-b27a-05c86be1ebff None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:49:27.504890 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0380b06-68c8-43ed-b27a-05c86be1ebff None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:49:27.506327 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0380b06-68c8-43ed-b27a-05c86be1ebff None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:49:27.520677 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-d0380b06-68c8-43ed-b27a-05c86be1ebff None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jun 23 15:49:27.520677 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:49:27.520677 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jun 23 15:49:27.520677 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jun 23 15:49:27.520677 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 15:49:27.520677 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 15:49:27.520677 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 15:49:27.520677 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 15:49:27.520677 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jun 23 15:49:27.520677 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Jun 23 15:49:27.520677 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 15:49:27.520677 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Jun 23 15:49:27.520677 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:49:27.520677 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:49:27.520677 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:49:27.520677 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:49:27.520677 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:49:27.521891 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:49:27.521891 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:49:27.521891 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:49:27.521891 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:49:27.521891 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:49:27.521891 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:49:27.521891 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 15:49:27.521891 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 15:49:27.521891 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 15:49:27.521891 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 15:49:27.521891 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:49:27.521891 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:49:27.521891 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:49:27.521891 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:49:27.521891 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:49:27.521891 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:49:27.523112 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:49:27.523112 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Jun 23 15:49:27.523112 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:49:27.523112 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:49:27.523112 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:49:27.523112 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:49:27.523112 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:49:27.523112 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:49:27.523112 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:49:27.523112 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:49:27.523112 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jun 23 15:49:27.523112 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 15:49:27.523112 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jun 23 15:49:27.523112 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Jun 23 15:49:27.523112 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 601/1165] 10.222.0.46 () {56 vars in 1117 bytes} [Thu Jun 23 15:49:27 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:49:27.528037 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c688d460-a2f0-40c5-b185-c20d087b494f None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:49:27.591079 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:49:27.625204 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c688d460-a2f0-40c5-b185-c20d087b494f None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:49:27.658652 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670128fa-3926-4930-92e5-04f22b1f421d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:49:27.659348 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670128fa-3926-4930-92e5-04f22b1f421d None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:49:27.659628 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670128fa-3926-4930-92e5-04f22b1f421d None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:49:27.659900 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670128fa-3926-4930-92e5-04f22b1f421d None admin] PATH_INFO: `/v3/domains` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:49:27.660616 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670128fa-3926-4930-92e5-04f22b1f421d None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:49:27.660921 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670128fa-3926-4930-92e5-04f22b1f421d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:49:27.662269 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670128fa-3926-4930-92e5-04f22b1f421d None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:49:27.669189 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 565/1166] 10.222.0.46 () {56 vars in 1115 bytes} [Thu Jun 23 15:49:27 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:49:29.699085 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde98513-0482-498e-be5d-bf0b0f6e8e2e None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:49:29.699611 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde98513-0482-498e-be5d-bf0b0f6e8e2e None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:49:29.699611 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde98513-0482-498e-be5d-bf0b0f6e8e2e None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:49:29.700982 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 602/1167] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jun 23 15:49:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 15:49:29.706313 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d25faeb-8dbd-49ea-880f-218862f3c498 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:49:29.706746 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d25faeb-8dbd-49ea-880f-218862f3c498 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:49:29.706900 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d25faeb-8dbd-49ea-880f-218862f3c498 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:49:29.723122 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-8d25faeb-8dbd-49ea-880f-218862f3c498 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:49:29.724296 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-8d25faeb-8dbd-49ea-880f-218862f3c498 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:49:29.788250 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 566/1168] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jun 23 15:49:29 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 15:49:29.811214 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde98513-0482-498e-be5d-bf0b0f6e8e2e None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:49:29.825608 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:49:29.913949 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde98513-0482-498e-be5d-bf0b0f6e8e2e None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:49:29.947024 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f0ae72-0f1f-4785-aba5-ed0d11398979 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:49:29.947682 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f0ae72-0f1f-4785-aba5-ed0d11398979 None admin] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:49:29.947967 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f0ae72-0f1f-4785-aba5-ed0d11398979 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:49:29.948243 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f0ae72-0f1f-4785-aba5-ed0d11398979 None admin] PATH_INFO: `/v3/domains` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:49:29.948990 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f0ae72-0f1f-4785-aba5-ed0d11398979 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:49:29.950358 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f0ae72-0f1f-4785-aba5-ed0d11398979 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:49:29.964339 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 603/1169] 10.222.0.46 () {60 vars in 1138 bytes} [Thu Jun 23 15:49:29 2022] POST /identity/v3/domains => generated 267 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 15:49:32.043003 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77eb826-693e-4497-a6f2-b5a8a606558c None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:49:32.043601 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77eb826-693e-4497-a6f2-b5a8a606558c None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:49:32.043601 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77eb826-693e-4497-a6f2-b5a8a606558c None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:49:32.044926 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 567/1170] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jun 23 15:49:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 15:49:32.050735 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac693ae-75b5-45d8-80d9-5347c17f4214 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:49:32.051181 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac693ae-75b5-45d8-80d9-5347c17f4214 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:49:32.051285 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac693ae-75b5-45d8-80d9-5347c17f4214 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:49:32.068366 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-0ac693ae-75b5-45d8-80d9-5347c17f4214 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:49:32.069543 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-0ac693ae-75b5-45d8-80d9-5347c17f4214 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:49:32.130665 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:49:32.136127 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 604/1171] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jun 23 15:49:32 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 15:49:32.159992 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77eb826-693e-4497-a6f2-b5a8a606558c None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:49:32.219339 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:49:32.260592 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77eb826-693e-4497-a6f2-b5a8a606558c None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:49:32.294503 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bed5bfd-6f4d-4455-a846-8c98c146653d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:49:32.295337 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bed5bfd-6f4d-4455-a846-8c98c146653d None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:49:32.295618 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bed5bfd-6f4d-4455-a846-8c98c146653d None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:49:32.295887 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bed5bfd-6f4d-4455-a846-8c98c146653d None admin] PATH_INFO: `/v3/domains/f55fac6129164fbeb4ccda7634c94a7e` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:49:32.306496 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bed5bfd-6f4d-4455-a846-8c98c146653d None admin] RBAC: Authorizing `identity:get_domain(domain_id=f55fac6129164fbeb4ccda7634c94a7e)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:49:32.307960 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bed5bfd-6f4d-4455-a846-8c98c146653d None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:49:32.309577 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 568/1172] 10.222.0.46 () {56 vars in 1183 bytes} [Thu Jun 23 15:49:32 2022] GET /identity/v3/domains/f55fac6129164fbeb4ccda7634c94a7e => generated 267 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:49:32.315574 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac693ae-75b5-45d8-80d9-5347c17f4214 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:49:32.412174 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac693ae-75b5-45d8-80d9-5347c17f4214 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:49:32.445223 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f68274-6f7f-4007-8142-fd61f1c496b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:49:32.445876 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f68274-6f7f-4007-8142-fd61f1c496b2 None admin] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:49:32.446164 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f68274-6f7f-4007-8142-fd61f1c496b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:49:32.446448 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f68274-6f7f-4007-8142-fd61f1c496b2 None admin] PATH_INFO: `/v3/projects` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:49:32.447183 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f68274-6f7f-4007-8142-fd61f1c496b2 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:49:32.448617 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f68274-6f7f-4007-8142-fd61f1c496b2 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:49:32.473328 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 605/1173] 10.222.0.46 () {60 vars in 1141 bytes} [Thu Jun 23 15:49:32 2022] POST /identity/v3/projects => generated 361 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 15:49:34.542471 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af770206-e035-4c57-9b0a-6059fc2b599d None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:49:34.543058 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af770206-e035-4c57-9b0a-6059fc2b599d None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:49:34.543058 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af770206-e035-4c57-9b0a-6059fc2b599d None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:49:34.544420 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 569/1174] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jun 23 15:49:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 15:49:34.550163 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beca5819-c051-41ef-8a83-b86958a8f2f5 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:49:34.550554 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beca5819-c051-41ef-8a83-b86958a8f2f5 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:49:34.550759 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beca5819-c051-41ef-8a83-b86958a8f2f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:49:34.567551 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-beca5819-c051-41ef-8a83-b86958a8f2f5 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:49:34.568747 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-beca5819-c051-41ef-8a83-b86958a8f2f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:49:34.615364 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:49:34.635727 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 606/1175] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jun 23 15:49:34 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 15:49:34.661071 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af770206-e035-4c57-9b0a-6059fc2b599d None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:49:34.759600 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af770206-e035-4c57-9b0a-6059fc2b599d None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:49:34.778851 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:49:34.793024 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb0cf8e-075f-4f3a-97dd-c1d78117492f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:49:34.793830 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb0cf8e-075f-4f3a-97dd-c1d78117492f None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:49:34.794113 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb0cf8e-075f-4f3a-97dd-c1d78117492f None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:49:34.794386 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb0cf8e-075f-4f3a-97dd-c1d78117492f None admin] PATH_INFO: `/v3/domains/f55fac6129164fbeb4ccda7634c94a7e` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:49:34.795578 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb0cf8e-075f-4f3a-97dd-c1d78117492f None admin] RBAC: Authorizing `identity:get_domain(domain_id=f55fac6129164fbeb4ccda7634c94a7e)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:49:34.797003 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb0cf8e-075f-4f3a-97dd-c1d78117492f None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:49:34.798629 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 570/1176] 10.222.0.46 () {56 vars in 1183 bytes} [Thu Jun 23 15:49:34 2022] GET /identity/v3/domains/f55fac6129164fbeb4ccda7634c94a7e => generated 267 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:49:34.804402 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beca5819-c051-41ef-8a83-b86958a8f2f5 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:49:34.903119 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beca5819-c051-41ef-8a83-b86958a8f2f5 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:49:34.935914 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ba930a-74d9-41e5-9bdf-457f275c9fee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:49:34.936571 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ba930a-74d9-41e5-9bdf-457f275c9fee None admin] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:49:34.936851 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ba930a-74d9-41e5-9bdf-457f275c9fee None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:49:34.937124 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ba930a-74d9-41e5-9bdf-457f275c9fee None admin] PATH_INFO: `/v3/users` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:49:34.937820 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ba930a-74d9-41e5-9bdf-457f275c9fee None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:49:34.939287 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ba930a-74d9-41e5-9bdf-457f275c9fee None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:49:34.958744 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 607/1177] 10.222.0.46 () {60 vars in 1132 bytes} [Thu Jun 23 15:49:34 2022] POST /identity/v3/users => generated 311 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 15:49:37.039593 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4957e8aa-6fbd-495f-af8d-b9bfecffcc24 None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:49:37.040148 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4957e8aa-6fbd-495f-af8d-b9bfecffcc24 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:49:37.040148 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4957e8aa-6fbd-495f-af8d-b9bfecffcc24 None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:49:37.041522 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 571/1178] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jun 23 15:49:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 15:49:37.046860 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53ee698-ee63-4023-b346-1c586e58759a None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:49:37.047252 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53ee698-ee63-4023-b346-1c586e58759a None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:49:37.047423 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53ee698-ee63-4023-b346-1c586e58759a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:49:37.063840 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-e53ee698-ee63-4023-b346-1c586e58759a None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:49:37.065007 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-e53ee698-ee63-4023-b346-1c586e58759a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:49:37.093542 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:49:37.131436 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 608/1179] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jun 23 15:49:37 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 15:49:37.155340 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4957e8aa-6fbd-495f-af8d-b9bfecffcc24 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:49:37.259461 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4957e8aa-6fbd-495f-af8d-b9bfecffcc24 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:49:37.292794 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005e3416-033f-4708-9989-5ce7e94dea48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:49:37.293529 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005e3416-033f-4708-9989-5ce7e94dea48 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:49:37.293816 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005e3416-033f-4708-9989-5ce7e94dea48 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:49:37.294095 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005e3416-033f-4708-9989-5ce7e94dea48 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:49:37.305166 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005e3416-033f-4708-9989-5ce7e94dea48 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 15:49:37.312726 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-005e3416-033f-4708-9989-5ce7e94dea48 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 15:49:37.312726 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:49:37.312726 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:49:37.312726 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:49:37.312726 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:49:37.312726 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:49:37.312726 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:49:37.312726 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:49:37.312726 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:49:37.312726 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:49:37.312726 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:49:37.312726 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:49:37.312726 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 15:49:37.312726 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 15:49:37.312726 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 15:49:37.312726 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 15:49:37.314020 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 15:49:37.314020 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 15:49:37.314020 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 15:49:37.314020 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 15:49:37.314020 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:49:37.314020 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:49:37.314020 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:49:37.314020 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:49:37.314020 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:49:37.314020 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:49:37.314020 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:49:37.314020 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Jun 23 15:49:37.314020 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:49:37.314020 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:49:37.314020 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:49:37.314020 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:49:37.315372 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:49:37.315372 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:49:37.315372 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:49:37.315372 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:49:37.315372 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 15:49:37.315372 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 15:49:37.315372 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 15:49:37.315372 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 15:49:37.315372 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 15:49:37.315372 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 15:49:37.315372 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 15:49:37.315372 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Jun 23 15:49:37.315372 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 572/1180] 10.222.0.46 () {56 vars in 1096 bytes} [Thu Jun 23 15:49:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:49:37.320066 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53ee698-ee63-4023-b346-1c586e58759a None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:49:37.417860 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53ee698-ee63-4023-b346-1c586e58759a None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:49:37.451124 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aca82ff-ea32-4541-a377-a081a5e5630c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:49:37.451786 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aca82ff-ea32-4541-a377-a081a5e5630c None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:49:37.452072 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aca82ff-ea32-4541-a377-a081a5e5630c None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:49:37.452352 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aca82ff-ea32-4541-a377-a081a5e5630c None admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:49:37.453101 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aca82ff-ea32-4541-a377-a081a5e5630c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:49:37.453411 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aca82ff-ea32-4541-a377-a081a5e5630c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:49:37.455428 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aca82ff-ea32-4541-a377-a081a5e5630c None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:49:37.463363 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 609/1181] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Jun 23 15:49:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:49:37.469088 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005e3416-033f-4708-9989-5ce7e94dea48 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:49:37.517267 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:49:37.568071 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005e3416-033f-4708-9989-5ce7e94dea48 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:49:37.601343 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313489d9-a23f-46af-a7f8-895bb9d20752 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:49:37.602231 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313489d9-a23f-46af-a7f8-895bb9d20752 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:49:37.602518 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313489d9-a23f-46af-a7f8-895bb9d20752 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:49:37.602833 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313489d9-a23f-46af-a7f8-895bb9d20752 None admin] PATH_INFO: `/v3/users/8638335c75454c9eb2a961bddeeb2606` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:49:37.615736 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313489d9-a23f-46af-a7f8-895bb9d20752 None admin] RBAC: Authorizing `identity:get_user(user_id=8638335c75454c9eb2a961bddeeb2606)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:49:37.617172 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313489d9-a23f-46af-a7f8-895bb9d20752 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:49:37.618906 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 573/1182] 10.222.0.46 () {56 vars in 1177 bytes} [Thu Jun 23 15:49:37 2022] GET /identity/v3/users/8638335c75454c9eb2a961bddeeb2606 => generated 311 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:49:37.624600 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aca82ff-ea32-4541-a377-a081a5e5630c None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:49:37.655723 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:49:37.722865 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aca82ff-ea32-4541-a377-a081a5e5630c None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:49:37.757886 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdc7c26-e344-4a88-8223-fd36ab694cb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:49:37.758535 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdc7c26-e344-4a88-8223-fd36ab694cb9 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:49:37.758853 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdc7c26-e344-4a88-8223-fd36ab694cb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:49:37.759135 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdc7c26-e344-4a88-8223-fd36ab694cb9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:49:37.826548 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdc7c26-e344-4a88-8223-fd36ab694cb9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:49:37.827955 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdc7c26-e344-4a88-8223-fd36ab694cb9 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:49:37.855524 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:49:37.924549 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 610/1183] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Jun 23 15:49:37 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 23 15:49:37.930360 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313489d9-a23f-46af-a7f8-895bb9d20752 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:49:38.026375 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313489d9-a23f-46af-a7f8-895bb9d20752 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:49:38.033491 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:49:38.060196 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e10754b-35b6-45db-b3be-a88250436a05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:49:38.061069 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e10754b-35b6-45db-b3be-a88250436a05 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:49:38.061355 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e10754b-35b6-45db-b3be-a88250436a05 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:49:38.061632 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e10754b-35b6-45db-b3be-a88250436a05 None admin] PATH_INFO: `/v3/projects/53d193fed72f4852a2148a916244b638` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:49:38.062841 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e10754b-35b6-45db-b3be-a88250436a05 None admin] RBAC: Authorizing `identity:get_project(project_id=53d193fed72f4852a2148a916244b638)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:49:38.064847 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e10754b-35b6-45db-b3be-a88250436a05 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:49:38.067208 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 574/1184] 10.222.0.46 () {56 vars in 1186 bytes} [Thu Jun 23 15:49:37 2022] GET /identity/v3/projects/53d193fed72f4852a2148a916244b638 => generated 361 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:49:38.073414 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdc7c26-e344-4a88-8223-fd36ab694cb9 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:49:38.170670 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdc7c26-e344-4a88-8223-fd36ab694cb9 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:49:38.198753 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:49:38.205056 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88498cf-e7f2-46bc-92a2-d1f5a0c9b2ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:49:38.205714 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88498cf-e7f2-46bc-92a2-d1f5a0c9b2ac None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:49:38.206000 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88498cf-e7f2-46bc-92a2-d1f5a0c9b2ac None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:49:38.206276 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88498cf-e7f2-46bc-92a2-d1f5a0c9b2ac None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:49:38.207143 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88498cf-e7f2-46bc-92a2-d1f5a0c9b2ac None admin] RBAC: Adding query filter params (role.id=812d3c567c52469cb2b35709fbdc857c, scope.project.id=53d193fed72f4852a2148a916244b638, user.id=8638335c75454c9eb2a961bddeeb2606) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:49:38.207457 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88498cf-e7f2-46bc-92a2-d1f5a0c9b2ac None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:49:38.208890 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88498cf-e7f2-46bc-92a2-d1f5a0c9b2ac None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:49:38.215075 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 611/1185] 10.222.0.46 () {56 vars in 1374 bytes} [Thu Jun 23 15:49:38 2022] GET /identity/v3/role_assignments?user.id=8638335c75454c9eb2a961bddeeb2606&scope.project.id=53d193fed72f4852a2148a916244b638&role.id=812d3c567c52469cb2b35709fbdc857c => generated 259 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:49:40.217144 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41813e0-3dd2-49b0-bb45-ab9644674fce None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:49:40.217684 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41813e0-3dd2-49b0-bb45-ab9644674fce None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:49:40.217684 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41813e0-3dd2-49b0-bb45-ab9644674fce None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:49:40.219106 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 575/1186] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jun 23 15:49:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 15:49:40.224337 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace95dcd-7db7-4924-bd79-48303ff27f78 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:49:40.224771 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace95dcd-7db7-4924-bd79-48303ff27f78 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:49:40.224900 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace95dcd-7db7-4924-bd79-48303ff27f78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:49:40.241989 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-ace95dcd-7db7-4924-bd79-48303ff27f78 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:49:40.243292 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-ace95dcd-7db7-4924-bd79-48303ff27f78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:49:40.309300 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 612/1187] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jun 23 15:49:40 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 15:49:40.332673 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41813e0-3dd2-49b0-bb45-ab9644674fce None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:49:40.437695 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41813e0-3dd2-49b0-bb45-ab9644674fce None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:49:40.470203 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111d7f7e-e966-49c2-84eb-c97a1e572566 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:49:40.470955 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111d7f7e-e966-49c2-84eb-c97a1e572566 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:49:40.471242 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111d7f7e-e966-49c2-84eb-c97a1e572566 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:49:40.471522 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111d7f7e-e966-49c2-84eb-c97a1e572566 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:49:40.481437 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111d7f7e-e966-49c2-84eb-c97a1e572566 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 15:49:40.489715 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-111d7f7e-e966-49c2-84eb-c97a1e572566 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 15:49:40.489715 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:49:40.489715 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:49:40.489715 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:49:40.489715 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:49:40.489715 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:49:40.489715 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:49:40.489715 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:49:40.489715 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:49:40.489715 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:49:40.489715 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:49:40.489715 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:49:40.489715 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 15:49:40.489715 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 15:49:40.489715 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 15:49:40.489715 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 15:49:40.490850 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 15:49:40.490850 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 15:49:40.490850 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 15:49:40.490850 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 15:49:40.490850 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:49:40.490850 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:49:40.490850 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:49:40.490850 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:49:40.490850 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:49:40.490850 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:49:40.490850 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:49:40.490850 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Jun 23 15:49:40.490850 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:49:40.490850 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:49:40.490850 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:49:40.490850 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:49:40.491732 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:49:40.491732 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:49:40.491732 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:49:40.491732 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:49:40.491732 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 15:49:40.491732 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 15:49:40.491732 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 15:49:40.491732 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 15:49:40.491732 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 15:49:40.491732 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 15:49:40.491732 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 15:49:40.491732 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Jun 23 15:49:40.491732 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 576/1188] 10.222.0.46 () {56 vars in 1096 bytes} [Thu Jun 23 15:49:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:49:40.497085 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace95dcd-7db7-4924-bd79-48303ff27f78 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:49:40.595226 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace95dcd-7db7-4924-bd79-48303ff27f78 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:49:40.618721 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:49:40.629354 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b25aa6-4ff7-42e9-bde5-f01067ead107 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:49:40.630024 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b25aa6-4ff7-42e9-bde5-f01067ead107 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:49:40.630316 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b25aa6-4ff7-42e9-bde5-f01067ead107 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:49:40.630637 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b25aa6-4ff7-42e9-bde5-f01067ead107 None admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:49:40.631418 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b25aa6-4ff7-42e9-bde5-f01067ead107 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:49:40.631730 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b25aa6-4ff7-42e9-bde5-f01067ead107 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:49:40.633108 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b25aa6-4ff7-42e9-bde5-f01067ead107 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:49:40.641062 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 613/1189] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Jun 23 15:49:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:49:40.646928 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111d7f7e-e966-49c2-84eb-c97a1e572566 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:49:40.681641 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:49:40.745662 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111d7f7e-e966-49c2-84eb-c97a1e572566 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:49:40.783565 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feedd102-5470-4cad-aff3-fae37f8a5a22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:49:40.784453 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feedd102-5470-4cad-aff3-fae37f8a5a22 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:49:40.784737 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feedd102-5470-4cad-aff3-fae37f8a5a22 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:49:40.785016 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feedd102-5470-4cad-aff3-fae37f8a5a22 None admin] PATH_INFO: `/v3/users/8638335c75454c9eb2a961bddeeb2606` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:49:40.786869 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feedd102-5470-4cad-aff3-fae37f8a5a22 None admin] RBAC: Authorizing `identity:get_user(user_id=8638335c75454c9eb2a961bddeeb2606)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:49:40.788328 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feedd102-5470-4cad-aff3-fae37f8a5a22 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:49:40.789983 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 577/1190] 10.222.0.46 () {56 vars in 1177 bytes} [Thu Jun 23 15:49:40 2022] GET /identity/v3/users/8638335c75454c9eb2a961bddeeb2606 => generated 311 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:49:40.796118 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b25aa6-4ff7-42e9-bde5-f01067ead107 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:49:40.894147 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b25aa6-4ff7-42e9-bde5-f01067ead107 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:49:40.928002 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4289f5a-f219-4c1e-bce7-b205200b683f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:49:40.928861 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4289f5a-f219-4c1e-bce7-b205200b683f None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:49:40.929141 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4289f5a-f219-4c1e-bce7-b205200b683f None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:49:40.929416 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4289f5a-f219-4c1e-bce7-b205200b683f None admin] PATH_INFO: `/v3/projects/53d193fed72f4852a2148a916244b638` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:49:40.930572 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4289f5a-f219-4c1e-bce7-b205200b683f None admin] RBAC: Authorizing `identity:get_project(project_id=53d193fed72f4852a2148a916244b638)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:49:40.932026 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4289f5a-f219-4c1e-bce7-b205200b683f None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:49:40.933708 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 614/1191] 10.222.0.46 () {56 vars in 1186 bytes} [Thu Jun 23 15:49:40 2022] GET /identity/v3/projects/53d193fed72f4852a2148a916244b638 => generated 361 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:49:40.939710 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feedd102-5470-4cad-aff3-fae37f8a5a22 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:49:41.036317 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feedd102-5470-4cad-aff3-fae37f8a5a22 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:49:41.037715 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:49:41.069977 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13341980-5bd5-4d62-9127-1c6145f97c47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:49:41.070890 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13341980-5bd5-4d62-9127-1c6145f97c47 None admin] REQUEST_METHOD: `PUT` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:49:41.071177 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13341980-5bd5-4d62-9127-1c6145f97c47 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:49:41.071447 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13341980-5bd5-4d62-9127-1c6145f97c47 None admin] PATH_INFO: `/v3/projects/53d193fed72f4852a2148a916244b638/users/8638335c75454c9eb2a961bddeeb2606/roles/812d3c567c52469cb2b35709fbdc857c` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:49:41.073364 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13341980-5bd5-4d62-9127-1c6145f97c47 None admin] RBAC: Authorizing `identity:create_grant(project_id=53d193fed72f4852a2148a916244b638, user_id=8638335c75454c9eb2a961bddeeb2606, role_id=812d3c567c52469cb2b35709fbdc857c)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:49:41.074821 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13341980-5bd5-4d62-9127-1c6145f97c47 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:49:41.087288 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 578/1192] 10.222.0.46 () {58 vars in 1439 bytes} [Thu Jun 23 15:49:40 2022] PUT /identity/v3/projects/53d193fed72f4852a2148a916244b638/users/8638335c75454c9eb2a961bddeeb2606/roles/812d3c567c52469cb2b35709fbdc857c => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 15:49:43.100489 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35294ffb-6fe3-4d44-ab61-1112949f64d9 None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:49:43.101042 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35294ffb-6fe3-4d44-ab61-1112949f64d9 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:49:43.101136 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35294ffb-6fe3-4d44-ab61-1112949f64d9 None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:49:43.102487 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 615/1193] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jun 23 15:49:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 15:49:43.107598 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba0642b-d5db-4fbf-be95-576976a9d610 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:49:43.108148 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba0642b-d5db-4fbf-be95-576976a9d610 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:49:43.108148 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba0642b-d5db-4fbf-be95-576976a9d610 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:49:43.125275 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-1ba0642b-d5db-4fbf-be95-576976a9d610 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:49:43.126467 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-1ba0642b-d5db-4fbf-be95-576976a9d610 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:49:43.190635 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 579/1194] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jun 23 15:49:43 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 15:49:43.214879 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35294ffb-6fe3-4d44-ab61-1112949f64d9 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:49:43.271450 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:49:43.317175 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35294ffb-6fe3-4d44-ab61-1112949f64d9 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:49:43.351127 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909de20f-9aca-4650-9edf-4150fc61ffb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:49:43.351853 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909de20f-9aca-4650-9edf-4150fc61ffb1 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:49:43.352137 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909de20f-9aca-4650-9edf-4150fc61ffb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:49:43.352417 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909de20f-9aca-4650-9edf-4150fc61ffb1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:49:43.362420 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909de20f-9aca-4650-9edf-4150fc61ffb1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 15:49:43.370037 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-909de20f-9aca-4650-9edf-4150fc61ffb1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 15:49:43.370037 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:49:43.370037 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:49:43.370037 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:49:43.370037 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:49:43.370037 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:49:43.370037 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:49:43.370037 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:49:43.370037 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:49:43.370037 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:49:43.370037 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:49:43.370037 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:49:43.370037 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 15:49:43.370037 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 15:49:43.370037 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 15:49:43.370037 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 15:49:43.371160 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 15:49:43.371160 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 15:49:43.371160 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 15:49:43.371160 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 15:49:43.371160 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:49:43.371160 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:49:43.371160 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:49:43.371160 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:49:43.371160 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:49:43.371160 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:49:43.371160 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:49:43.371160 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Jun 23 15:49:43.371160 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:49:43.371160 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:49:43.371160 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:49:43.371160 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:49:43.372068 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:49:43.372068 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:49:43.372068 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:49:43.372068 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:49:43.372068 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 15:49:43.372068 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 15:49:43.372068 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 15:49:43.372068 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 15:49:43.372068 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 15:49:43.372068 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 15:49:43.372068 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 15:49:43.372068 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Jun 23 15:49:43.372068 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 616/1195] 10.222.0.46 () {56 vars in 1096 bytes} [Thu Jun 23 15:49:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:49:43.377765 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba0642b-d5db-4fbf-be95-576976a9d610 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:49:43.465709 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:49:43.475227 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba0642b-d5db-4fbf-be95-576976a9d610 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:49:43.508103 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc3c6c1-c200-4210-9b55-1ebd38a08be2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:49:43.508763 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc3c6c1-c200-4210-9b55-1ebd38a08be2 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:49:43.509053 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc3c6c1-c200-4210-9b55-1ebd38a08be2 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:49:43.509323 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc3c6c1-c200-4210-9b55-1ebd38a08be2 None admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:49:43.510074 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc3c6c1-c200-4210-9b55-1ebd38a08be2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:49:43.510380 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc3c6c1-c200-4210-9b55-1ebd38a08be2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:49:43.511773 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc3c6c1-c200-4210-9b55-1ebd38a08be2 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:49:43.520304 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 580/1196] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Jun 23 15:49:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:49:43.526282 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909de20f-9aca-4650-9edf-4150fc61ffb1 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:49:43.624681 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909de20f-9aca-4650-9edf-4150fc61ffb1 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:49:43.638362 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:49:43.659306 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459de8cc-37fb-45e2-9ae2-7d90fc72505e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:49:43.660177 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459de8cc-37fb-45e2-9ae2-7d90fc72505e None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:49:43.660456 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459de8cc-37fb-45e2-9ae2-7d90fc72505e None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:49:43.660727 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459de8cc-37fb-45e2-9ae2-7d90fc72505e None admin] PATH_INFO: `/v3/users/8638335c75454c9eb2a961bddeeb2606` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:49:43.661907 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459de8cc-37fb-45e2-9ae2-7d90fc72505e None admin] RBAC: Authorizing `identity:get_user(user_id=8638335c75454c9eb2a961bddeeb2606)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:49:43.663376 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459de8cc-37fb-45e2-9ae2-7d90fc72505e None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:49:43.664989 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 617/1197] 10.222.0.46 () {56 vars in 1177 bytes} [Thu Jun 23 15:49:43 2022] GET /identity/v3/users/8638335c75454c9eb2a961bddeeb2606 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:49:43.670802 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc3c6c1-c200-4210-9b55-1ebd38a08be2 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:49:43.774827 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc3c6c1-c200-4210-9b55-1ebd38a08be2 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:49:43.807283 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af284869-774d-4216-97b6-6e04b025213b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:49:43.807934 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af284869-774d-4216-97b6-6e04b025213b None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:49:43.808220 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af284869-774d-4216-97b6-6e04b025213b None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:49:43.808499 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af284869-774d-4216-97b6-6e04b025213b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:49:43.831879 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:49:43.874078 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af284869-774d-4216-97b6-6e04b025213b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:49:43.876147 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af284869-774d-4216-97b6-6e04b025213b None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:49:43.969672 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 581/1198] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Jun 23 15:49:43 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 23 15:49:43.975685 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459de8cc-37fb-45e2-9ae2-7d90fc72505e None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:49:44.075089 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459de8cc-37fb-45e2-9ae2-7d90fc72505e None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:49:44.107401 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7aad5c-8512-4819-9141-05008499a79f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:49:44.108262 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7aad5c-8512-4819-9141-05008499a79f None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:49:44.108546 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7aad5c-8512-4819-9141-05008499a79f None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:49:44.108823 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7aad5c-8512-4819-9141-05008499a79f None admin] PATH_INFO: `/v3/projects/53d193fed72f4852a2148a916244b638` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:49:44.110517 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7aad5c-8512-4819-9141-05008499a79f None admin] RBAC: Authorizing `identity:get_project(project_id=53d193fed72f4852a2148a916244b638)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:49:44.111983 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7aad5c-8512-4819-9141-05008499a79f None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:49:44.113672 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 618/1199] 10.222.0.46 () {56 vars in 1186 bytes} [Thu Jun 23 15:49:43 2022] GET /identity/v3/projects/53d193fed72f4852a2148a916244b638 => generated 361 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:49:44.119784 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af284869-774d-4216-97b6-6e04b025213b None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:49:44.174070 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:49:44.217063 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af284869-774d-4216-97b6-6e04b025213b None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:49:44.250291 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3345dc36-a1f9-4969-a41b-cb584e9731ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:49:44.251009 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3345dc36-a1f9-4969-a41b-cb584e9731ae None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:49:44.251294 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3345dc36-a1f9-4969-a41b-cb584e9731ae None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:49:44.251571 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3345dc36-a1f9-4969-a41b-cb584e9731ae None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:49:44.252401 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3345dc36-a1f9-4969-a41b-cb584e9731ae None admin] RBAC: Adding query filter params (role.id=812d3c567c52469cb2b35709fbdc857c, scope.project.id=53d193fed72f4852a2148a916244b638, user.id=8638335c75454c9eb2a961bddeeb2606) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:49:44.252712 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3345dc36-a1f9-4969-a41b-cb584e9731ae None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:49:44.254135 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3345dc36-a1f9-4969-a41b-cb584e9731ae None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:49:44.259847 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 582/1200] 10.222.0.46 () {56 vars in 1374 bytes} [Thu Jun 23 15:49:44 2022] GET /identity/v3/role_assignments?user.id=8638335c75454c9eb2a961bddeeb2606&scope.project.id=53d193fed72f4852a2148a916244b638&role.id=812d3c567c52469cb2b35709fbdc857c => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:49:54.103256 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=75 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:49:54.257306 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=41 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:50:40.059373 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c97ee86-c95b-41fb-9a93-84e56b408a7b None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:50:40.059373 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c97ee86-c95b-41fb-9a93-84e56b408a7b None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:50:40.060631 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c97ee86-c95b-41fb-9a93-84e56b408a7b None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:50:40.062610 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 619/1201] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jun 23 15:50:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 15:50:40.068377 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81337dae-3ce7-4991-a585-63f5fd5be160 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:50:40.069111 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81337dae-3ce7-4991-a585-63f5fd5be160 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:50:40.069227 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81337dae-3ce7-4991-a585-63f5fd5be160 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:50:40.088813 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-81337dae-3ce7-4991-a585-63f5fd5be160 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:50:40.090029 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-81337dae-3ce7-4991-a585-63f5fd5be160 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:50:40.194754 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 583/1202] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jun 23 15:50:40 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 15:50:40.222959 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c97ee86-c95b-41fb-9a93-84e56b408a7b None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:50:40.368700 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c97ee86-c95b-41fb-9a93-84e56b408a7b None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:50:40.417384 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3ef292-64fc-414b-8d3a-26067eddedaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:50:40.418223 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3ef292-64fc-414b-8d3a-26067eddedaa None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:50:40.418510 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3ef292-64fc-414b-8d3a-26067eddedaa None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:50:40.418876 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3ef292-64fc-414b-8d3a-26067eddedaa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:50:40.432723 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3ef292-64fc-414b-8d3a-26067eddedaa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:50:40.434306 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3ef292-64fc-414b-8d3a-26067eddedaa None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:50:40.454919 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-7b3ef292-64fc-414b-8d3a-26067eddedaa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 15:50:40.454919 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:50:40.454919 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:50:40.454919 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:50:40.454919 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:50:40.454919 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:50:40.454919 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:50:40.454919 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:50:40.454919 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:50:40.454919 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:50:40.454919 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:50:40.454919 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:50:40.454919 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 15:50:40.454919 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 15:50:40.454919 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 15:50:40.455995 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 15:50:40.455995 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:50:40.455995 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:50:40.455995 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:50:40.455995 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:50:40.455995 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:50:40.455995 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:50:40.455995 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:50:40.455995 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Jun 23 15:50:40.455995 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:50:40.455995 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:50:40.455995 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:50:40.455995 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:50:40.455995 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:50:40.455995 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:50:40.455995 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:50:40.455995 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:50:40.457020 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 15:50:40.457020 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 15:50:40.457020 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 15:50:40.457020 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Jun 23 15:50:40.461602 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 620/1203] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Jun 23 15:50:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:50:40.465243 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81337dae-3ce7-4991-a585-63f5fd5be160 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:50:40.612136 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81337dae-3ce7-4991-a585-63f5fd5be160 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:50:40.653473 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:50:40.660247 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517556a4-9f84-44b6-a044-60ea500440a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:50:40.660961 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517556a4-9f84-44b6-a044-60ea500440a9 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:50:40.661787 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517556a4-9f84-44b6-a044-60ea500440a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:50:40.662461 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517556a4-9f84-44b6-a044-60ea500440a9 None admin] PATH_INFO: `/v3/domains` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:50:40.663480 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517556a4-9f84-44b6-a044-60ea500440a9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:50:40.664139 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517556a4-9f84-44b6-a044-60ea500440a9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:50:40.665999 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517556a4-9f84-44b6-a044-60ea500440a9 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:50:40.689457 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 584/1204] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Jun 23 15:50:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:50:40.690913 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3ef292-64fc-414b-8d3a-26067eddedaa None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:50:40.758004 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:50:40.826231 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3ef292-64fc-414b-8d3a-26067eddedaa None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:50:40.874921 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29f0ffd-396e-46e7-a6f6-bcf7d67e333b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:50:40.875653 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29f0ffd-396e-46e7-a6f6-bcf7d67e333b None admin] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:50:40.875948 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29f0ffd-396e-46e7-a6f6-bcf7d67e333b None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:50:40.876223 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29f0ffd-396e-46e7-a6f6-bcf7d67e333b None admin] PATH_INFO: `/v3/users` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:50:40.877071 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29f0ffd-396e-46e7-a6f6-bcf7d67e333b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:50:40.878995 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29f0ffd-396e-46e7-a6f6-bcf7d67e333b None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:50:40.903815 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 621/1205] 10.222.0.46 () {60 vars in 1132 bytes} [Thu Jun 23 15:50:40 2022] POST /identity/v3/users => generated 251 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 15:50:43.102305 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef4a2c9-fe0f-4898-8e05-cc2003fb8135 None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:50:43.102935 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef4a2c9-fe0f-4898-8e05-cc2003fb8135 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:50:43.102935 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef4a2c9-fe0f-4898-8e05-cc2003fb8135 None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:50:43.104658 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 585/1206] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jun 23 15:50:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 15:50:43.111341 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f9b4ff-0928-47ed-8491-2a097501a156 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:50:43.111837 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f9b4ff-0928-47ed-8491-2a097501a156 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:50:43.111925 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f9b4ff-0928-47ed-8491-2a097501a156 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:50:43.133145 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-a4f9b4ff-0928-47ed-8491-2a097501a156 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:50:43.134645 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-a4f9b4ff-0928-47ed-8491-2a097501a156 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:50:43.230417 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 622/1207] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jun 23 15:50:43 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 23 15:50:43.255346 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef4a2c9-fe0f-4898-8e05-cc2003fb8135 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:50:43.395979 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef4a2c9-fe0f-4898-8e05-cc2003fb8135 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:50:43.437308 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5397959-5cc2-4a64-aebe-a3ce020e04f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:50:43.438152 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5397959-5cc2-4a64-aebe-a3ce020e04f6 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:50:43.438502 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5397959-5cc2-4a64-aebe-a3ce020e04f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:50:43.438745 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5397959-5cc2-4a64-aebe-a3ce020e04f6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:50:43.450732 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5397959-5cc2-4a64-aebe-a3ce020e04f6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 15:50:43.460460 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-c5397959-5cc2-4a64-aebe-a3ce020e04f6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 23 15:50:43.460460 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:50:43.460460 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:50:43.460460 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:50:43.460460 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:50:43.460460 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:50:43.460460 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:50:43.460460 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:50:43.460460 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:50:43.460460 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:50:43.460460 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:50:43.460460 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:50:43.460460 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 15:50:43.460460 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 15:50:43.460460 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 15:50:43.460460 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 15:50:43.461472 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 15:50:43.461472 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 15:50:43.461472 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 15:50:43.461472 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 15:50:43.461472 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:50:43.461472 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:50:43.461472 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:50:43.461472 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:50:43.461472 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:50:43.461472 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:50:43.461472 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:50:43.461472 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Jun 23 15:50:43.461472 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:50:43.461472 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:50:43.461472 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:50:43.461472 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:50:43.462346 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:50:43.462346 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:50:43.462346 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:50:43.462346 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:50:43.462346 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 15:50:43.462346 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 15:50:43.462346 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 15:50:43.462346 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 15:50:43.462346 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 15:50:43.462346 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 15:50:43.462346 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 23 15:50:43.462346 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Jun 23 15:50:43.467167 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 586/1208] 10.222.0.46 () {56 vars in 1102 bytes} [Thu Jun 23 15:50:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:50:43.468736 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f9b4ff-0928-47ed-8491-2a097501a156 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:50:43.515583 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:50:43.589216 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f9b4ff-0928-47ed-8491-2a097501a156 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:50:43.632400 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5d35dc-1196-4b7f-ba10-b88d1e092898 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:50:43.633531 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5d35dc-1196-4b7f-ba10-b88d1e092898 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:50:43.633648 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5d35dc-1196-4b7f-ba10-b88d1e092898 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:50:43.633910 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5d35dc-1196-4b7f-ba10-b88d1e092898 None admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:50:43.635014 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5d35dc-1196-4b7f-ba10-b88d1e092898 None admin] RBAC: Adding query filter params (name=service) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:50:43.635479 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5d35dc-1196-4b7f-ba10-b88d1e092898 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:50:43.637368 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5d35dc-1196-4b7f-ba10-b88d1e092898 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:50:43.648594 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 623/1209] 10.222.0.46 () {56 vars in 1103 bytes} [Thu Jun 23 15:50:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:50:43.654699 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5397959-5cc2-4a64-aebe-a3ce020e04f6 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:50:43.714878 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:50:43.774831 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5397959-5cc2-4a64-aebe-a3ce020e04f6 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:50:43.827105 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f71711-7e5a-4b06-b033-3c90e57dd652 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:50:43.829001 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f71711-7e5a-4b06-b033-3c90e57dd652 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:50:43.829324 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f71711-7e5a-4b06-b033-3c90e57dd652 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:50:43.829731 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f71711-7e5a-4b06-b033-3c90e57dd652 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:50:43.841788 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f71711-7e5a-4b06-b033-3c90e57dd652 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:50:43.843587 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f71711-7e5a-4b06-b033-3c90e57dd652 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:50:43.865123 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-73f71711-7e5a-4b06-b033-3c90e57dd652 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 15:50:43.865123 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:50:43.865123 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:50:43.865123 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:50:43.865123 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:50:43.865123 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:50:43.865123 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:50:43.865123 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:50:43.865123 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:50:43.865123 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:50:43.865123 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:50:43.865123 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:50:43.865123 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 15:50:43.865123 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 15:50:43.865123 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 15:50:43.866182 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 15:50:43.866182 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:50:43.866182 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:50:43.866182 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:50:43.866182 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:50:43.866182 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:50:43.866182 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:50:43.866182 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:50:43.866182 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Jun 23 15:50:43.866182 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:50:43.866182 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:50:43.866182 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:50:43.866182 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:50:43.866182 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:50:43.866182 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:50:43.866182 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:50:43.866182 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:50:43.867437 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 15:50:43.867437 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 15:50:43.867437 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 15:50:43.867437 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Jun 23 15:50:43.867437 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 587/1210] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Jun 23 15:50:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 15:50:43.872182 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5d35dc-1196-4b7f-ba10-b88d1e092898 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:50:43.992575 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:50:43.995875 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5d35dc-1196-4b7f-ba10-b88d1e092898 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:50:44.046652 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d5e28a-8c11-416f-af4c-60d435420c5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:50:44.047783 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d5e28a-8c11-416f-af4c-60d435420c5d None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:50:44.048112 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d5e28a-8c11-416f-af4c-60d435420c5d None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:50:44.048441 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d5e28a-8c11-416f-af4c-60d435420c5d None admin] PATH_INFO: `/v3/domains` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:50:44.049436 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d5e28a-8c11-416f-af4c-60d435420c5d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:50:44.049794 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d5e28a-8c11-416f-af4c-60d435420c5d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:50:44.051693 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d5e28a-8c11-416f-af4c-60d435420c5d None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:50:44.071940 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 624/1211] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Jun 23 15:50:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:50:44.078077 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f71711-7e5a-4b06-b033-3c90e57dd652 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:50:44.203626 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f71711-7e5a-4b06-b033-3c90e57dd652 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:50:44.206367 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:50:44.249297 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee9ea1b-2cc4-45db-8be4-c5f1da148172 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:50:44.250379 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee9ea1b-2cc4-45db-8be4-c5f1da148172 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:50:44.250761 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee9ea1b-2cc4-45db-8be4-c5f1da148172 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:50:44.251213 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee9ea1b-2cc4-45db-8be4-c5f1da148172 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:50:44.261383 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee9ea1b-2cc4-45db-8be4-c5f1da148172 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:50:44.263313 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee9ea1b-2cc4-45db-8be4-c5f1da148172 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:50:44.284604 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-3ee9ea1b-2cc4-45db-8be4-c5f1da148172 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 23 15:50:44.284604 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:50:44.284604 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:50:44.284604 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:50:44.284604 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:50:44.284604 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:50:44.284604 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:50:44.284604 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:50:44.284604 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:50:44.284604 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:50:44.284604 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:50:44.284604 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:50:44.284604 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 15:50:44.284604 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 15:50:44.284604 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 15:50:44.284604 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 15:50:44.288187 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:50:44.288187 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:50:44.288187 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 15:50:44.288187 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 15:50:44.288187 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 15:50:44.288187 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 15:50:44.288187 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:50:44.288187 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:50:44.288187 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:50:44.288187 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:50:44.288187 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:50:44.288187 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Jun 23 15:50:44.288187 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:50:44.288187 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:50:44.288187 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:50:44.288187 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:50:44.288187 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:50:44.288187 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:50:44.289552 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:50:44.289552 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:50:44.289552 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 15:50:44.289552 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 15:50:44.289552 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 15:50:44.289552 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 15:50:44.289552 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 15:50:44.289552 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 15:50:44.289552 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 23 15:50:44.289552 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Jun 23 15:50:44.289552 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 588/1212] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Jun 23 15:50:44 2022] GET /identity/v3/users/placement => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:50:44.294066 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d5e28a-8c11-416f-af4c-60d435420c5d None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:50:44.417494 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d5e28a-8c11-416f-af4c-60d435420c5d None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:50:44.466992 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338b6056-da93-415d-be12-cb1f772ac492 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:50:44.467765 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338b6056-da93-415d-be12-cb1f772ac492 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:50:44.468016 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338b6056-da93-415d-be12-cb1f772ac492 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:50:44.468376 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338b6056-da93-415d-be12-cb1f772ac492 None admin] PATH_INFO: `/v3/users` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:50:44.469161 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338b6056-da93-415d-be12-cb1f772ac492 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:50:44.469475 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338b6056-da93-415d-be12-cb1f772ac492 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:50:44.470941 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338b6056-da93-415d-be12-cb1f772ac492 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:50:44.491999 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 625/1213] 10.222.0.46 () {56 vars in 1143 bytes} [Thu Jun 23 15:50:44 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:50:44.498216 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee9ea1b-2cc4-45db-8be4-c5f1da148172 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:50:44.626732 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee9ea1b-2cc4-45db-8be4-c5f1da148172 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:50:44.671764 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84dd1754-b778-4002-bcb3-3b0054fbb5c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:50:44.672763 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84dd1754-b778-4002-bcb3-3b0054fbb5c9 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:50:44.673118 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84dd1754-b778-4002-bcb3-3b0054fbb5c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:50:44.673465 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84dd1754-b778-4002-bcb3-3b0054fbb5c9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:50:44.707352 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:50:44.750324 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84dd1754-b778-4002-bcb3-3b0054fbb5c9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:50:44.751838 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84dd1754-b778-4002-bcb3-3b0054fbb5c9 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:50:44.886192 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 589/1214] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Jun 23 15:50:44 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 23 15:50:44.892100 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338b6056-da93-415d-be12-cb1f772ac492 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:50:44.916963 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:50:45.017194 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338b6056-da93-415d-be12-cb1f772ac492 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:50:45.058653 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a957f9-1778-42cd-ac85-bb97c6e99981 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:50:45.060052 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a957f9-1778-42cd-ac85-bb97c6e99981 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:50:45.060412 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a957f9-1778-42cd-ac85-bb97c6e99981 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:50:45.060760 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a957f9-1778-42cd-ac85-bb97c6e99981 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:50:45.073735 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a957f9-1778-42cd-ac85-bb97c6e99981 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:50:45.075331 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a957f9-1778-42cd-ac85-bb97c6e99981 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:50:45.098963 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-e9a957f9-1778-42cd-ac85-bb97c6e99981 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 15:50:45.098963 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:50:45.098963 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:50:45.098963 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:50:45.098963 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:50:45.098963 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:50:45.098963 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:50:45.098963 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:50:45.098963 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:50:45.098963 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:50:45.098963 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:50:45.098963 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:50:45.098963 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 15:50:45.098963 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 15:50:45.098963 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 15:50:45.100096 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 15:50:45.100096 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:50:45.100096 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:50:45.100096 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:50:45.100096 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:50:45.100096 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:50:45.100096 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:50:45.100096 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:50:45.100096 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Jun 23 15:50:45.100096 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:50:45.100096 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:50:45.100096 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:50:45.100096 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:50:45.100096 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:50:45.100096 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:50:45.100096 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:50:45.100096 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:50:45.105656 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 15:50:45.105656 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 15:50:45.105656 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 15:50:45.105656 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Jun 23 15:50:45.105656 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 626/1215] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Jun 23 15:50:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 15:50:45.105999 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84dd1754-b778-4002-bcb3-3b0054fbb5c9 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:50:45.173396 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:50:45.218890 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84dd1754-b778-4002-bcb3-3b0054fbb5c9 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:50:45.260702 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77047500-b2fb-45d2-9bab-9beeb6a84082 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:50:45.261651 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77047500-b2fb-45d2-9bab-9beeb6a84082 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:50:45.262091 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77047500-b2fb-45d2-9bab-9beeb6a84082 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:50:45.262513 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77047500-b2fb-45d2-9bab-9beeb6a84082 None admin] PATH_INFO: `/v3/domains` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:50:45.263963 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77047500-b2fb-45d2-9bab-9beeb6a84082 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:50:45.265269 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77047500-b2fb-45d2-9bab-9beeb6a84082 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:50:45.265643 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77047500-b2fb-45d2-9bab-9beeb6a84082 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:50:45.281512 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 590/1216] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Jun 23 15:50:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:50:45.287695 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a957f9-1778-42cd-ac85-bb97c6e99981 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:50:45.369913 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:50:45.410207 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a957f9-1778-42cd-ac85-bb97c6e99981 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:50:45.453680 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d4e463-3d3d-4bd3-af89-07964072958b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:50:45.454409 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d4e463-3d3d-4bd3-af89-07964072958b None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:50:45.454719 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d4e463-3d3d-4bd3-af89-07964072958b None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:50:45.455008 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d4e463-3d3d-4bd3-af89-07964072958b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:50:45.462664 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d4e463-3d3d-4bd3-af89-07964072958b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:50:45.464152 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d4e463-3d3d-4bd3-af89-07964072958b None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:50:45.484216 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-80d4e463-3d3d-4bd3-af89-07964072958b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 15:50:45.484216 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:50:45.484216 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:50:45.484216 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:50:45.484216 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:50:45.484216 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:50:45.484216 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:50:45.484216 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:50:45.484216 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:50:45.484216 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:50:45.484216 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:50:45.484216 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:50:45.484216 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 15:50:45.484216 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 15:50:45.484216 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 15:50:45.485416 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 15:50:45.485416 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:50:45.485416 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:50:45.485416 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:50:45.485416 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:50:45.485416 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:50:45.485416 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:50:45.485416 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:50:45.485416 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Jun 23 15:50:45.485416 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:50:45.485416 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:50:45.485416 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:50:45.485416 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:50:45.485416 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:50:45.485416 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:50:45.485416 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:50:45.485416 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:50:45.491625 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 15:50:45.491625 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 15:50:45.491625 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 15:50:45.491625 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 15:50:45.491625 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 15:50:45.491625 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 15:50:45.491625 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 15:50:45.491625 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Jun 23 15:50:45.491625 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 627/1217] 10.222.0.46 () {56 vars in 1111 bytes} [Thu Jun 23 15:50:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:50:45.492233 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77047500-b2fb-45d2-9bab-9beeb6a84082 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:50:45.616108 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77047500-b2fb-45d2-9bab-9beeb6a84082 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:50:45.644785 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:50:45.658732 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29e64eb-1034-455a-a802-63fab53f0f7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:50:45.659426 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29e64eb-1034-455a-a802-63fab53f0f7e None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:50:45.659799 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29e64eb-1034-455a-a802-63fab53f0f7e None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:50:45.660019 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29e64eb-1034-455a-a802-63fab53f0f7e None admin] PATH_INFO: `/v3/projects` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:50:45.660901 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29e64eb-1034-455a-a802-63fab53f0f7e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:50:45.661239 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29e64eb-1034-455a-a802-63fab53f0f7e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:50:45.663409 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29e64eb-1034-455a-a802-63fab53f0f7e None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:50:45.676559 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 591/1218] 10.222.0.46 () {56 vars in 1148 bytes} [Thu Jun 23 15:50:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:50:45.682932 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d4e463-3d3d-4bd3-af89-07964072958b None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:50:45.805456 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d4e463-3d3d-4bd3-af89-07964072958b None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:50:45.838999 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:50:45.846217 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11aaa2a-0b99-4527-b846-72e8d7f9ac48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:50:45.847065 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11aaa2a-0b99-4527-b846-72e8d7f9ac48 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:50:45.847506 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11aaa2a-0b99-4527-b846-72e8d7f9ac48 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:50:45.847889 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11aaa2a-0b99-4527-b846-72e8d7f9ac48 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:50:45.848878 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11aaa2a-0b99-4527-b846-72e8d7f9ac48 None admin] RBAC: Adding query filter params (role.id=bfa8ecadc3d5483e821420aa0f0fb41e, scope.project.id=28ecdf5b474c44098c722f677d2a0b77, user.id=3f66988b73644fe38b9849663ee70f25) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:50:45.849241 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11aaa2a-0b99-4527-b846-72e8d7f9ac48 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:50:45.850798 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11aaa2a-0b99-4527-b846-72e8d7f9ac48 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:50:45.857458 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 628/1219] 10.222.0.46 () {56 vars in 1374 bytes} [Thu Jun 23 15:50:45 2022] GET /identity/v3/role_assignments?user.id=3f66988b73644fe38b9849663ee70f25&scope.project.id=28ecdf5b474c44098c722f677d2a0b77&role.id=bfa8ecadc3d5483e821420aa0f0fb41e => generated 259 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:50:47.929879 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6569c2-a676-420a-b98a-73bfe29d2841 None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:50:47.930448 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6569c2-a676-420a-b98a-73bfe29d2841 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:50:47.930448 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6569c2-a676-420a-b98a-73bfe29d2841 None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:50:47.931843 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 592/1220] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jun 23 15:50:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 15:50:47.938241 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850c3ae9-7729-47f3-b117-ef50d3398a4c None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:50:47.938785 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850c3ae9-7729-47f3-b117-ef50d3398a4c None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:50:47.938876 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850c3ae9-7729-47f3-b117-ef50d3398a4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:50:47.956932 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-850c3ae9-7729-47f3-b117-ef50d3398a4c None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:50:47.958078 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-850c3ae9-7729-47f3-b117-ef50d3398a4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:50:48.030421 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 629/1221] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jun 23 15:50:47 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 15:50:48.053147 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6569c2-a676-420a-b98a-73bfe29d2841 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:50:48.166024 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6569c2-a676-420a-b98a-73bfe29d2841 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:50:48.211527 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955cb4aa-65f2-4445-a27f-88e37357de55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:50:48.212372 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955cb4aa-65f2-4445-a27f-88e37357de55 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:50:48.212659 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955cb4aa-65f2-4445-a27f-88e37357de55 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:50:48.212938 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955cb4aa-65f2-4445-a27f-88e37357de55 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:50:48.226878 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955cb4aa-65f2-4445-a27f-88e37357de55 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 15:50:48.237007 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-955cb4aa-65f2-4445-a27f-88e37357de55 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 23 15:50:48.237007 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:50:48.237007 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:50:48.237007 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:50:48.237007 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:50:48.237007 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:50:48.237007 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:50:48.237007 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:50:48.237007 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:50:48.237007 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:50:48.237007 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:50:48.237007 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:50:48.237007 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 15:50:48.237007 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 15:50:48.237007 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 15:50:48.237007 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 15:50:48.239778 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 15:50:48.239778 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 15:50:48.239778 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 15:50:48.239778 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 15:50:48.239778 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:50:48.239778 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:50:48.239778 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:50:48.239778 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:50:48.239778 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:50:48.239778 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:50:48.239778 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:50:48.239778 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Jun 23 15:50:48.239778 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:50:48.239778 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:50:48.239778 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:50:48.239778 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:50:48.241620 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:50:48.241620 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:50:48.241620 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:50:48.241620 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:50:48.241620 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 15:50:48.241620 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 15:50:48.241620 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 15:50:48.241620 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 15:50:48.241620 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 15:50:48.241620 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 15:50:48.241620 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 23 15:50:48.241620 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Jun 23 15:50:48.241620 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 593/1222] 10.222.0.46 () {56 vars in 1102 bytes} [Thu Jun 23 15:50:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:50:48.245845 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850c3ae9-7729-47f3-b117-ef50d3398a4c None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:50:48.370806 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850c3ae9-7729-47f3-b117-ef50d3398a4c None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:50:48.404527 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:50:48.421411 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3154c51-f961-405e-b0ea-4de192db6263 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:50:48.422304 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3154c51-f961-405e-b0ea-4de192db6263 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:50:48.422588 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3154c51-f961-405e-b0ea-4de192db6263 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:50:48.422934 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3154c51-f961-405e-b0ea-4de192db6263 None admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:50:48.423775 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3154c51-f961-405e-b0ea-4de192db6263 None admin] RBAC: Adding query filter params (name=service) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:50:48.424145 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3154c51-f961-405e-b0ea-4de192db6263 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:50:48.425888 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3154c51-f961-405e-b0ea-4de192db6263 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:50:48.439589 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 630/1223] 10.222.0.46 () {56 vars in 1103 bytes} [Thu Jun 23 15:50:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:50:48.445769 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955cb4aa-65f2-4445-a27f-88e37357de55 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:50:48.501547 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:50:48.573166 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955cb4aa-65f2-4445-a27f-88e37357de55 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:50:48.614548 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fc5c46-2438-46ca-84a1-96ce11bd06cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:50:48.615482 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fc5c46-2438-46ca-84a1-96ce11bd06cc None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:50:48.615763 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fc5c46-2438-46ca-84a1-96ce11bd06cc None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:50:48.616041 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fc5c46-2438-46ca-84a1-96ce11bd06cc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:50:48.628208 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fc5c46-2438-46ca-84a1-96ce11bd06cc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:50:48.629673 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fc5c46-2438-46ca-84a1-96ce11bd06cc None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:50:48.647972 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-a1fc5c46-2438-46ca-84a1-96ce11bd06cc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 15:50:48.647972 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:50:48.647972 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:50:48.647972 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:50:48.647972 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:50:48.647972 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:50:48.647972 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:50:48.647972 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:50:48.647972 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:50:48.647972 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:50:48.647972 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:50:48.647972 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:50:48.647972 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 15:50:48.647972 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 15:50:48.647972 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 15:50:48.649152 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 15:50:48.649152 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:50:48.649152 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:50:48.649152 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:50:48.649152 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:50:48.649152 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:50:48.649152 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:50:48.649152 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:50:48.649152 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Jun 23 15:50:48.649152 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:50:48.649152 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:50:48.649152 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:50:48.649152 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:50:48.649152 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:50:48.649152 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:50:48.649152 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:50:48.649152 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:50:48.650285 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 15:50:48.650285 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 15:50:48.650285 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 15:50:48.650285 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Jun 23 15:50:48.650285 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 594/1224] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Jun 23 15:50:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:50:48.656196 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3154c51-f961-405e-b0ea-4de192db6263 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:50:48.765296 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3154c51-f961-405e-b0ea-4de192db6263 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:50:48.801535 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c253be-3dca-4dc9-a045-c6f40e37a082 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:50:48.802205 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c253be-3dca-4dc9-a045-c6f40e37a082 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:50:48.802486 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c253be-3dca-4dc9-a045-c6f40e37a082 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:50:48.802781 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c253be-3dca-4dc9-a045-c6f40e37a082 None admin] PATH_INFO: `/v3/domains` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:50:48.803515 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c253be-3dca-4dc9-a045-c6f40e37a082 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:50:48.803818 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c253be-3dca-4dc9-a045-c6f40e37a082 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:50:48.805194 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c253be-3dca-4dc9-a045-c6f40e37a082 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:50:48.826025 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 631/1225] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Jun 23 15:50:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:50:48.827423 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fc5c46-2438-46ca-84a1-96ce11bd06cc None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:50:48.930167 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:50:48.937351 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fc5c46-2438-46ca-84a1-96ce11bd06cc None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:50:48.971967 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a1a083-99a0-4ccf-8f92-d404ce5af2a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:50:48.972789 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a1a083-99a0-4ccf-8f92-d404ce5af2a9 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:50:48.973071 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a1a083-99a0-4ccf-8f92-d404ce5af2a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:50:48.973339 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a1a083-99a0-4ccf-8f92-d404ce5af2a9 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:50:48.981583 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a1a083-99a0-4ccf-8f92-d404ce5af2a9 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:50:48.983723 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a1a083-99a0-4ccf-8f92-d404ce5af2a9 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:50:48.999187 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-68a1a083-99a0-4ccf-8f92-d404ce5af2a9 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 23 15:50:48.999187 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:50:48.999187 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:50:48.999187 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:50:48.999187 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:50:48.999187 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:50:48.999187 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:50:48.999187 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:50:48.999187 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:50:48.999187 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:50:48.999187 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:50:48.999187 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:50:48.999187 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 15:50:48.999187 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 15:50:48.999187 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 15:50:48.999187 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 15:50:49.000253 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:50:49.000253 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:50:49.000253 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 15:50:49.000253 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 15:50:49.000253 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 15:50:49.000253 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 15:50:49.000253 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:50:49.000253 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:50:49.000253 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:50:49.000253 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:50:49.000253 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:50:49.000253 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Jun 23 15:50:49.000253 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:50:49.000253 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:50:49.000253 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:50:49.000253 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:50:49.000253 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:50:49.000253 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:50:49.001517 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:50:49.001517 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:50:49.001517 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 15:50:49.001517 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 15:50:49.001517 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 15:50:49.001517 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 15:50:49.001517 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 15:50:49.001517 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 15:50:49.001517 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 23 15:50:49.001517 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Jun 23 15:50:49.001517 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 595/1226] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Jun 23 15:50:48 2022] GET /identity/v3/users/placement => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:50:49.006302 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c253be-3dca-4dc9-a045-c6f40e37a082 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:50:49.057654 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:50:49.114741 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c253be-3dca-4dc9-a045-c6f40e37a082 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:50:49.151990 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbbb444-8a53-4251-af2e-970e6ce6f87f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:50:49.152770 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbbb444-8a53-4251-af2e-970e6ce6f87f None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:50:49.153063 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbbb444-8a53-4251-af2e-970e6ce6f87f None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:50:49.153376 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbbb444-8a53-4251-af2e-970e6ce6f87f None admin] PATH_INFO: `/v3/users` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:50:49.154959 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbbb444-8a53-4251-af2e-970e6ce6f87f None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:50:49.155330 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbbb444-8a53-4251-af2e-970e6ce6f87f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:50:49.156980 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbbb444-8a53-4251-af2e-970e6ce6f87f None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:50:49.180209 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 632/1227] 10.222.0.46 () {56 vars in 1143 bytes} [Thu Jun 23 15:50:49 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:50:49.186065 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a1a083-99a0-4ccf-8f92-d404ce5af2a9 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:50:49.322150 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a1a083-99a0-4ccf-8f92-d404ce5af2a9 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:50:49.356862 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f62ccff-433d-443f-87f2-d5bd7e109d54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:50:49.357674 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f62ccff-433d-443f-87f2-d5bd7e109d54 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:50:49.357959 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f62ccff-433d-443f-87f2-d5bd7e109d54 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:50:49.358234 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f62ccff-433d-443f-87f2-d5bd7e109d54 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:50:49.369205 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f62ccff-433d-443f-87f2-d5bd7e109d54 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:50:49.370657 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f62ccff-433d-443f-87f2-d5bd7e109d54 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:50:49.389783 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-6f62ccff-433d-443f-87f2-d5bd7e109d54 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 15:50:49.389783 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:50:49.389783 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:50:49.389783 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:50:49.389783 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:50:49.389783 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:50:49.389783 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:50:49.389783 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:50:49.389783 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:50:49.389783 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:50:49.389783 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:50:49.389783 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:50:49.389783 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 15:50:49.389783 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 15:50:49.389783 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 15:50:49.391213 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 15:50:49.391213 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:50:49.391213 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:50:49.391213 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:50:49.391213 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:50:49.391213 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:50:49.391213 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:50:49.391213 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:50:49.391213 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Jun 23 15:50:49.391213 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:50:49.391213 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:50:49.391213 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:50:49.391213 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:50:49.391213 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:50:49.391213 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:50:49.391213 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:50:49.391213 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:50:49.392122 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 15:50:49.392122 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 15:50:49.392122 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 15:50:49.392122 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Jun 23 15:50:49.392122 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 596/1228] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Jun 23 15:50:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:50:49.397167 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbbb444-8a53-4251-af2e-970e6ce6f87f None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:50:49.512316 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbbb444-8a53-4251-af2e-970e6ce6f87f None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:50:49.515329 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:50:49.549672 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e990e7-eb06-4c06-bf92-d9f703673e99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:50:49.550343 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e990e7-eb06-4c06-bf92-d9f703673e99 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:50:49.550624 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e990e7-eb06-4c06-bf92-d9f703673e99 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:50:49.551002 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e990e7-eb06-4c06-bf92-d9f703673e99 None admin] PATH_INFO: `/v3/domains` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:50:49.551684 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e990e7-eb06-4c06-bf92-d9f703673e99 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:50:49.551988 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e990e7-eb06-4c06-bf92-d9f703673e99 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:50:49.553382 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e990e7-eb06-4c06-bf92-d9f703673e99 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:50:49.567900 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 633/1229] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Jun 23 15:50:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 15:50:49.573779 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f62ccff-433d-443f-87f2-d5bd7e109d54 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:50:49.584042 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:50:49.687611 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f62ccff-433d-443f-87f2-d5bd7e109d54 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:50:49.727597 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ad5ac0-62fc-4076-9fbb-22fc1359ebe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:50:49.728501 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ad5ac0-62fc-4076-9fbb-22fc1359ebe4 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:50:49.728787 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ad5ac0-62fc-4076-9fbb-22fc1359ebe4 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:50:49.729068 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ad5ac0-62fc-4076-9fbb-22fc1359ebe4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:50:49.737400 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ad5ac0-62fc-4076-9fbb-22fc1359ebe4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:50:49.739010 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ad5ac0-62fc-4076-9fbb-22fc1359ebe4 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:50:49.755775 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-d0ad5ac0-62fc-4076-9fbb-22fc1359ebe4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 15:50:49.755775 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:50:49.755775 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:50:49.755775 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:50:49.755775 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:50:49.755775 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:50:49.755775 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:50:49.755775 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:50:49.755775 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:50:49.755775 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:50:49.755775 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:50:49.755775 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:50:49.755775 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 15:50:49.755775 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 15:50:49.755775 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 15:50:49.757011 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 15:50:49.757011 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:50:49.757011 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:50:49.757011 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:50:49.757011 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:50:49.757011 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:50:49.757011 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:50:49.757011 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:50:49.757011 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Jun 23 15:50:49.757011 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:50:49.757011 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:50:49.757011 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:50:49.757011 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:50:49.757011 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:50:49.757011 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:50:49.757011 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:50:49.757011 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:50:49.758076 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 15:50:49.758076 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 15:50:49.758076 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 15:50:49.758076 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 15:50:49.758076 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 15:50:49.758076 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 15:50:49.758076 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 15:50:49.758076 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Jun 23 15:50:49.758076 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 597/1230] 10.222.0.46 () {56 vars in 1111 bytes} [Thu Jun 23 15:50:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 15:50:49.763453 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e990e7-eb06-4c06-bf92-d9f703673e99 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:50:49.882609 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e990e7-eb06-4c06-bf92-d9f703673e99 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:50:49.922526 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65079217-e198-45fd-915c-36669203d4a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:50:49.923278 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65079217-e198-45fd-915c-36669203d4a0 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:50:49.923574 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65079217-e198-45fd-915c-36669203d4a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:50:49.923893 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65079217-e198-45fd-915c-36669203d4a0 None admin] PATH_INFO: `/v3/projects` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:50:49.924881 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65079217-e198-45fd-915c-36669203d4a0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:50:49.925243 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65079217-e198-45fd-915c-36669203d4a0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:50:49.926788 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65079217-e198-45fd-915c-36669203d4a0 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:50:49.939353 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 634/1231] 10.222.0.46 () {56 vars in 1148 bytes} [Thu Jun 23 15:50:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:50:49.945656 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ad5ac0-62fc-4076-9fbb-22fc1359ebe4 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:50:50.018722 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:50:50.058464 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ad5ac0-62fc-4076-9fbb-22fc1359ebe4 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:50:50.102147 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853494d1-23e5-4164-9abc-99f0c23744be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:50:50.103186 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853494d1-23e5-4164-9abc-99f0c23744be None admin] REQUEST_METHOD: `PUT` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:50:50.103475 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853494d1-23e5-4164-9abc-99f0c23744be None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:50:50.103751 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853494d1-23e5-4164-9abc-99f0c23744be None admin] PATH_INFO: `/v3/projects/28ecdf5b474c44098c722f677d2a0b77/users/3f66988b73644fe38b9849663ee70f25/roles/bfa8ecadc3d5483e821420aa0f0fb41e` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:50:50.120120 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853494d1-23e5-4164-9abc-99f0c23744be None admin] RBAC: Authorizing `identity:create_grant(project_id=28ecdf5b474c44098c722f677d2a0b77, user_id=3f66988b73644fe38b9849663ee70f25, role_id=bfa8ecadc3d5483e821420aa0f0fb41e)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:50:50.121627 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853494d1-23e5-4164-9abc-99f0c23744be None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:50:50.135652 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 598/1232] 10.222.0.46 () {58 vars in 1439 bytes} [Thu Jun 23 15:50:49 2022] PUT /identity/v3/projects/28ecdf5b474c44098c722f677d2a0b77/users/3f66988b73644fe38b9849663ee70f25/roles/bfa8ecadc3d5483e821420aa0f0fb41e => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 15:50:52.228480 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c0721a-ee8b-4355-9874-2da53a9fe39a None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:50:52.229052 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c0721a-ee8b-4355-9874-2da53a9fe39a None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:50:52.229052 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c0721a-ee8b-4355-9874-2da53a9fe39a None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:50:52.230392 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 635/1233] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jun 23 15:50:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 15:50:52.236544 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8924d90b-715c-4fda-997f-3c1841c644df None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:50:52.237003 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8924d90b-715c-4fda-997f-3c1841c644df None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:50:52.237127 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8924d90b-715c-4fda-997f-3c1841c644df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:50:52.254282 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-8924d90b-715c-4fda-997f-3c1841c644df None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:50:52.255538 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-8924d90b-715c-4fda-997f-3c1841c644df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:50:52.337825 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 599/1234] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jun 23 15:50:52 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 15:50:52.361961 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c0721a-ee8b-4355-9874-2da53a9fe39a None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:50:52.393080 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:50:52.486595 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c0721a-ee8b-4355-9874-2da53a9fe39a None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:50:52.524244 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b770ef20-9d66-4377-953b-95797da8e640 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:50:52.525052 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b770ef20-9d66-4377-953b-95797da8e640 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:50:52.525345 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b770ef20-9d66-4377-953b-95797da8e640 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:50:52.525624 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b770ef20-9d66-4377-953b-95797da8e640 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:50:52.538572 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b770ef20-9d66-4377-953b-95797da8e640 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 15:50:52.548615 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-b770ef20-9d66-4377-953b-95797da8e640 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 23 15:50:52.548615 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:50:52.548615 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:50:52.548615 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:50:52.548615 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:50:52.548615 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:50:52.548615 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:50:52.548615 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:50:52.548615 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:50:52.548615 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:50:52.548615 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:50:52.548615 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:50:52.548615 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 15:50:52.548615 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 15:50:52.548615 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 15:50:52.548615 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 15:50:52.549737 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 15:50:52.549737 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 15:50:52.549737 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 15:50:52.549737 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 15:50:52.549737 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:50:52.549737 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:50:52.549737 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:50:52.549737 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:50:52.549737 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:50:52.549737 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:50:52.549737 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:50:52.549737 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Jun 23 15:50:52.549737 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:50:52.549737 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:50:52.549737 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:50:52.549737 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:50:52.550632 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:50:52.550632 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:50:52.550632 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:50:52.550632 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:50:52.550632 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 15:50:52.550632 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 15:50:52.550632 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 15:50:52.550632 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 15:50:52.550632 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 15:50:52.550632 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 15:50:52.550632 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 23 15:50:52.550632 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Jun 23 15:50:52.550632 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 636/1235] 10.222.0.46 () {56 vars in 1102 bytes} [Thu Jun 23 15:50:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:50:52.556539 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8924d90b-715c-4fda-997f-3c1841c644df None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:50:52.609002 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:50:52.672255 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8924d90b-715c-4fda-997f-3c1841c644df None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:50:52.710056 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114c87a7-9e30-4a8a-aed2-8eb1e171a98b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:50:52.710770 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114c87a7-9e30-4a8a-aed2-8eb1e171a98b None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:50:52.711058 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114c87a7-9e30-4a8a-aed2-8eb1e171a98b None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:50:52.711353 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114c87a7-9e30-4a8a-aed2-8eb1e171a98b None admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:50:52.712197 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114c87a7-9e30-4a8a-aed2-8eb1e171a98b None admin] RBAC: Adding query filter params (name=service) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:50:52.712638 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114c87a7-9e30-4a8a-aed2-8eb1e171a98b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:50:52.714531 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114c87a7-9e30-4a8a-aed2-8eb1e171a98b None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:50:52.725039 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 600/1236] 10.222.0.46 () {56 vars in 1103 bytes} [Thu Jun 23 15:50:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:50:52.730804 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b770ef20-9d66-4377-953b-95797da8e640 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:50:52.823497 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:50:52.842263 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b770ef20-9d66-4377-953b-95797da8e640 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:50:52.879878 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47921726-23c5-4d4e-b3cf-8126d375a2da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:50:52.880757 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47921726-23c5-4d4e-b3cf-8126d375a2da None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:50:52.881043 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47921726-23c5-4d4e-b3cf-8126d375a2da None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:50:52.881319 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47921726-23c5-4d4e-b3cf-8126d375a2da None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:50:52.893804 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47921726-23c5-4d4e-b3cf-8126d375a2da None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:50:52.895550 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47921726-23c5-4d4e-b3cf-8126d375a2da None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:50:52.914344 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-47921726-23c5-4d4e-b3cf-8126d375a2da None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 15:50:52.914344 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:50:52.914344 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:50:52.914344 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:50:52.914344 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:50:52.914344 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:50:52.914344 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:50:52.914344 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:50:52.914344 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:50:52.914344 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:50:52.914344 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:50:52.914344 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:50:52.914344 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 15:50:52.914344 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 15:50:52.914344 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 15:50:52.915539 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 15:50:52.915539 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:50:52.915539 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:50:52.915539 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:50:52.915539 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:50:52.915539 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:50:52.915539 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:50:52.915539 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:50:52.915539 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Jun 23 15:50:52.915539 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:50:52.915539 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:50:52.915539 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:50:52.915539 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:50:52.915539 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:50:52.915539 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:50:52.915539 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:50:52.915539 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:50:52.922425 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 15:50:52.922425 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 15:50:52.922425 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 15:50:52.922425 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Jun 23 15:50:52.922425 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 637/1237] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Jun 23 15:50:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 15:50:52.922867 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114c87a7-9e30-4a8a-aed2-8eb1e171a98b None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:50:53.031979 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114c87a7-9e30-4a8a-aed2-8eb1e171a98b None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:50:53.043584 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:50:53.069928 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c96e6f-67d3-4fce-812c-a84eff79d7c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:50:53.070623 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c96e6f-67d3-4fce-812c-a84eff79d7c8 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:50:53.071270 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c96e6f-67d3-4fce-812c-a84eff79d7c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:50:53.071553 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c96e6f-67d3-4fce-812c-a84eff79d7c8 None admin] PATH_INFO: `/v3/domains` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:50:53.072287 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c96e6f-67d3-4fce-812c-a84eff79d7c8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:50:53.072592 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c96e6f-67d3-4fce-812c-a84eff79d7c8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:50:53.074019 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c96e6f-67d3-4fce-812c-a84eff79d7c8 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:50:53.086181 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 601/1238] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Jun 23 15:50:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:50:53.092666 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47921726-23c5-4d4e-b3cf-8126d375a2da None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:50:53.206598 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47921726-23c5-4d4e-b3cf-8126d375a2da None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:50:53.245234 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcd5e6f-3b60-494a-84fd-ac3f21bf722b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:50:53.246176 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcd5e6f-3b60-494a-84fd-ac3f21bf722b None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:50:53.246298 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcd5e6f-3b60-494a-84fd-ac3f21bf722b None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:50:53.246572 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcd5e6f-3b60-494a-84fd-ac3f21bf722b None admin] PATH_INFO: `/v3/users/placement` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:50:53.250070 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:50:53.256000 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbcd5e6f-3b60-494a-84fd-ac3f21bf722b None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:50:53.257708 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbcd5e6f-3b60-494a-84fd-ac3f21bf722b None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:50:53.273741 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-cbcd5e6f-3b60-494a-84fd-ac3f21bf722b None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 23 15:50:53.273741 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:50:53.273741 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:50:53.273741 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:50:53.273741 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:50:53.273741 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:50:53.273741 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:50:53.273741 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:50:53.273741 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:50:53.273741 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:50:53.273741 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:50:53.273741 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:50:53.273741 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 15:50:53.273741 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 15:50:53.273741 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 15:50:53.273741 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 15:50:53.275195 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:50:53.275195 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:50:53.275195 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 15:50:53.275195 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 15:50:53.275195 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 15:50:53.275195 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 15:50:53.275195 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:50:53.275195 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:50:53.275195 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:50:53.275195 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:50:53.275195 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:50:53.275195 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Jun 23 15:50:53.275195 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:50:53.275195 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:50:53.275195 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:50:53.275195 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:50:53.275195 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:50:53.275195 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:50:53.276374 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:50:53.276374 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:50:53.276374 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 15:50:53.276374 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 15:50:53.276374 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 15:50:53.276374 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 15:50:53.276374 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 15:50:53.276374 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 15:50:53.276374 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 23 15:50:53.276374 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Jun 23 15:50:53.276374 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 638/1239] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Jun 23 15:50:53 2022] GET /identity/v3/users/placement => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:50:53.281191 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c96e6f-67d3-4fce-812c-a84eff79d7c8 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:50:53.409776 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c96e6f-67d3-4fce-812c-a84eff79d7c8 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:50:53.453195 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1f93e0-dbb1-456f-acce-e230e8b70eea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:50:53.454029 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1f93e0-dbb1-456f-acce-e230e8b70eea None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:50:53.454451 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1f93e0-dbb1-456f-acce-e230e8b70eea None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:50:53.454780 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1f93e0-dbb1-456f-acce-e230e8b70eea None admin] PATH_INFO: `/v3/users` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:50:53.456756 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1f93e0-dbb1-456f-acce-e230e8b70eea None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:50:53.457075 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1f93e0-dbb1-456f-acce-e230e8b70eea None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:50:53.458550 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1f93e0-dbb1-456f-acce-e230e8b70eea None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:50:53.482743 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 602/1240] 10.222.0.46 () {56 vars in 1143 bytes} [Thu Jun 23 15:50:53 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 15:50:53.486129 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcd5e6f-3b60-494a-84fd-ac3f21bf722b None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:50:53.623043 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcd5e6f-3b60-494a-84fd-ac3f21bf722b None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:50:53.660886 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f920aba7-c8b2-4a47-a31d-c427348032fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:50:53.661585 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f920aba7-c8b2-4a47-a31d-c427348032fb None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:50:53.661977 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f920aba7-c8b2-4a47-a31d-c427348032fb None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:50:53.662225 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f920aba7-c8b2-4a47-a31d-c427348032fb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:50:53.740448 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:50:53.751408 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f920aba7-c8b2-4a47-a31d-c427348032fb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:50:53.753234 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f920aba7-c8b2-4a47-a31d-c427348032fb None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:50:53.868373 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 639/1241] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Jun 23 15:50:53 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 23 15:50:53.875191 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1f93e0-dbb1-456f-acce-e230e8b70eea None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:50:53.904795 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:50:53.996550 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1f93e0-dbb1-456f-acce-e230e8b70eea None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:50:54.034643 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4410e0ef-0058-4fd9-a876-8a9e71656a43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:50:54.035756 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4410e0ef-0058-4fd9-a876-8a9e71656a43 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:50:54.036074 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4410e0ef-0058-4fd9-a876-8a9e71656a43 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:50:54.036345 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4410e0ef-0058-4fd9-a876-8a9e71656a43 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:50:54.048189 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4410e0ef-0058-4fd9-a876-8a9e71656a43 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:50:54.049879 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4410e0ef-0058-4fd9-a876-8a9e71656a43 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:50:54.069324 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-4410e0ef-0058-4fd9-a876-8a9e71656a43 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 15:50:54.069324 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:50:54.069324 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:50:54.069324 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:50:54.069324 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:50:54.069324 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:50:54.069324 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:50:54.069324 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:50:54.069324 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:50:54.069324 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:50:54.069324 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:50:54.069324 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:50:54.069324 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 15:50:54.069324 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 15:50:54.069324 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 15:50:54.070330 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 15:50:54.070330 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:50:54.070330 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:50:54.070330 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:50:54.070330 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:50:54.070330 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:50:54.070330 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:50:54.070330 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:50:54.070330 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Jun 23 15:50:54.070330 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:50:54.070330 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:50:54.070330 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:50:54.070330 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:50:54.070330 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:50:54.070330 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:50:54.070330 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:50:54.070330 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:50:54.071259 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 15:50:54.071259 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 15:50:54.071259 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 15:50:54.071259 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Jun 23 15:50:54.071259 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 603/1242] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Jun 23 15:50:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:50:54.077318 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f920aba7-c8b2-4a47-a31d-c427348032fb None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:50:54.177873 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:50:54.185421 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f920aba7-c8b2-4a47-a31d-c427348032fb None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:50:54.226478 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4124e444-a844-4a90-8c5c-0cd4f6e3c353 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:50:54.227261 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4124e444-a844-4a90-8c5c-0cd4f6e3c353 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:50:54.228698 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4124e444-a844-4a90-8c5c-0cd4f6e3c353 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:50:54.228698 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4124e444-a844-4a90-8c5c-0cd4f6e3c353 None admin] PATH_INFO: `/v3/domains` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:50:54.228899 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4124e444-a844-4a90-8c5c-0cd4f6e3c353 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:50:54.229169 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4124e444-a844-4a90-8c5c-0cd4f6e3c353 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:50:54.230767 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4124e444-a844-4a90-8c5c-0cd4f6e3c353 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:50:54.246875 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 640/1243] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Jun 23 15:50:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:50:54.252599 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4410e0ef-0058-4fd9-a876-8a9e71656a43 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:50:54.355370 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:50:54.382360 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4410e0ef-0058-4fd9-a876-8a9e71656a43 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:50:54.420625 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e507158-27ef-4e6d-9818-d62eebd15ce3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:50:54.421415 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e507158-27ef-4e6d-9818-d62eebd15ce3 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:50:54.421692 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e507158-27ef-4e6d-9818-d62eebd15ce3 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:50:54.421966 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e507158-27ef-4e6d-9818-d62eebd15ce3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:50:54.431750 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e507158-27ef-4e6d-9818-d62eebd15ce3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:50:54.433231 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e507158-27ef-4e6d-9818-d62eebd15ce3 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:50:54.449199 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-1e507158-27ef-4e6d-9818-d62eebd15ce3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 15:50:54.449199 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:50:54.449199 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:50:54.449199 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:50:54.449199 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:50:54.449199 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:50:54.449199 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:50:54.449199 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:50:54.449199 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:50:54.449199 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:50:54.449199 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:50:54.449199 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:50:54.449199 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 15:50:54.449199 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 15:50:54.449199 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 15:50:54.450479 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 15:50:54.450479 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:50:54.450479 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:50:54.450479 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:50:54.450479 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:50:54.450479 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:50:54.450479 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:50:54.450479 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:50:54.450479 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Jun 23 15:50:54.450479 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:50:54.450479 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:50:54.450479 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:50:54.450479 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:50:54.450479 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:50:54.450479 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:50:54.450479 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:50:54.450479 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:50:54.451438 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 15:50:54.451438 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 15:50:54.451438 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 15:50:54.451438 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 15:50:54.451438 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 15:50:54.451438 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 15:50:54.451438 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 15:50:54.451438 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Jun 23 15:50:54.451438 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 604/1244] 10.222.0.46 () {56 vars in 1111 bytes} [Thu Jun 23 15:50:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:50:54.456671 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4124e444-a844-4a90-8c5c-0cd4f6e3c353 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:50:54.575032 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4124e444-a844-4a90-8c5c-0cd4f6e3c353 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:50:54.613974 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db6caae-5df8-4b3b-ae7b-18fd6e7c709c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:50:54.614755 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db6caae-5df8-4b3b-ae7b-18fd6e7c709c None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:50:54.615089 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db6caae-5df8-4b3b-ae7b-18fd6e7c709c None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:50:54.615351 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db6caae-5df8-4b3b-ae7b-18fd6e7c709c None admin] PATH_INFO: `/v3/projects` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:50:54.616295 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db6caae-5df8-4b3b-ae7b-18fd6e7c709c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:50:54.616638 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db6caae-5df8-4b3b-ae7b-18fd6e7c709c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:50:54.618097 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db6caae-5df8-4b3b-ae7b-18fd6e7c709c None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:50:54.632225 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 641/1245] 10.222.0.46 () {56 vars in 1148 bytes} [Thu Jun 23 15:50:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:50:54.638439 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e507158-27ef-4e6d-9818-d62eebd15ce3 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:50:54.758873 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e507158-27ef-4e6d-9818-d62eebd15ce3 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:50:54.796117 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d099518-795f-46d6-b71e-20426d2f9141 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:50:54.796794 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d099518-795f-46d6-b71e-20426d2f9141 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:50:54.797077 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d099518-795f-46d6-b71e-20426d2f9141 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:50:54.797358 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d099518-795f-46d6-b71e-20426d2f9141 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:50:54.798337 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d099518-795f-46d6-b71e-20426d2f9141 None admin] RBAC: Adding query filter params (role.id=bfa8ecadc3d5483e821420aa0f0fb41e, scope.project.id=28ecdf5b474c44098c722f677d2a0b77, user.id=3f66988b73644fe38b9849663ee70f25) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:50:54.798651 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d099518-795f-46d6-b71e-20426d2f9141 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:50:54.800144 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d099518-795f-46d6-b71e-20426d2f9141 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:50:54.806737 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 605/1246] 10.222.0.46 () {56 vars in 1374 bytes} [Thu Jun 23 15:50:54 2022] GET /identity/v3/role_assignments?user.id=3f66988b73644fe38b9849663ee70f25&scope.project.id=28ecdf5b474c44098c722f677d2a0b77&role.id=bfa8ecadc3d5483e821420aa0f0fb41e => generated 608 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:50:56.997606 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca0cfbe-dbea-4e90-ad66-78c3f24753ad None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:50:56.998482 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca0cfbe-dbea-4e90-ad66-78c3f24753ad None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:50:56.998482 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca0cfbe-dbea-4e90-ad66-78c3f24753ad None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:50:57.000810 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 642/1247] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jun 23 15:50:56 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 15:50:57.006453 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06da6d74-edbd-46da-a48a-7053186e930c None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:50:57.007428 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06da6d74-edbd-46da-a48a-7053186e930c None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:50:57.007428 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06da6d74-edbd-46da-a48a-7053186e930c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:50:57.012481 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:50:57.026544 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-06da6d74-edbd-46da-a48a-7053186e930c None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:50:57.027790 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-06da6d74-edbd-46da-a48a-7053186e930c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:50:57.117273 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 606/1248] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jun 23 15:50:57 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 23 15:50:57.144130 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca0cfbe-dbea-4e90-ad66-78c3f24753ad None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:50:57.171454 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:50:57.300793 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca0cfbe-dbea-4e90-ad66-78c3f24753ad None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:50:57.369659 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10573743-a6dd-42e9-827c-96078e72df1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:50:57.370541 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10573743-a6dd-42e9-827c-96078e72df1c None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:50:57.370870 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10573743-a6dd-42e9-827c-96078e72df1c None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:50:57.371107 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10573743-a6dd-42e9-827c-96078e72df1c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:50:57.383917 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10573743-a6dd-42e9-827c-96078e72df1c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 15:50:57.392761 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-10573743-a6dd-42e9-827c-96078e72df1c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 15:50:57.392761 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:50:57.392761 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:50:57.392761 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:50:57.392761 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:50:57.392761 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:50:57.392761 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:50:57.392761 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:50:57.392761 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:50:57.392761 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:50:57.392761 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:50:57.392761 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:50:57.392761 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 15:50:57.392761 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 15:50:57.392761 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 15:50:57.392761 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 15:50:57.394972 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 15:50:57.394972 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 15:50:57.394972 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 15:50:57.394972 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 15:50:57.394972 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:50:57.394972 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:50:57.394972 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:50:57.394972 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:50:57.394972 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:50:57.394972 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:50:57.394972 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:50:57.394972 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Jun 23 15:50:57.394972 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:50:57.394972 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:50:57.394972 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:50:57.394972 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:50:57.397339 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:50:57.397339 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:50:57.397339 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:50:57.397339 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:50:57.397339 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 15:50:57.397339 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 15:50:57.397339 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 15:50:57.397339 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 15:50:57.397339 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 15:50:57.397339 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 15:50:57.397339 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 15:50:57.397339 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Jun 23 15:50:57.397339 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 643/1249] 10.222.0.46 () {56 vars in 1096 bytes} [Thu Jun 23 15:50:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 15:50:57.403899 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06da6d74-edbd-46da-a48a-7053186e930c None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:50:57.545515 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06da6d74-edbd-46da-a48a-7053186e930c None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:50:57.591944 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:50:57.599273 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79f0d6b-0087-4836-9cba-ee833bc25b15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:50:57.600219 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79f0d6b-0087-4836-9cba-ee833bc25b15 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:50:57.600583 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79f0d6b-0087-4836-9cba-ee833bc25b15 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:50:57.600966 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79f0d6b-0087-4836-9cba-ee833bc25b15 None admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:50:57.602069 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79f0d6b-0087-4836-9cba-ee833bc25b15 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:50:57.602587 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79f0d6b-0087-4836-9cba-ee833bc25b15 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:50:57.604544 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79f0d6b-0087-4836-9cba-ee833bc25b15 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:50:57.618959 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 607/1250] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Jun 23 15:50:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:50:57.626328 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10573743-a6dd-42e9-827c-96078e72df1c None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:50:57.739770 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:50:57.774684 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10573743-a6dd-42e9-827c-96078e72df1c None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:50:57.823638 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983e6b9e-c8a2-452f-9486-8ea8684fbfbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:50:57.824640 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983e6b9e-c8a2-452f-9486-8ea8684fbfbd None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:50:57.825135 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983e6b9e-c8a2-452f-9486-8ea8684fbfbd None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:50:57.825586 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983e6b9e-c8a2-452f-9486-8ea8684fbfbd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:50:57.843809 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983e6b9e-c8a2-452f-9486-8ea8684fbfbd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:50:57.845577 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983e6b9e-c8a2-452f-9486-8ea8684fbfbd None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:50:57.873309 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-983e6b9e-c8a2-452f-9486-8ea8684fbfbd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 15:50:57.873309 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:50:57.873309 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:50:57.873309 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:50:57.873309 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:50:57.873309 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:50:57.873309 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:50:57.873309 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:50:57.873309 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:50:57.873309 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:50:57.873309 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:50:57.873309 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:50:57.873309 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 15:50:57.873309 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 15:50:57.873309 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 15:50:57.875076 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 15:50:57.875076 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:50:57.875076 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:50:57.875076 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:50:57.875076 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:50:57.875076 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:50:57.875076 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:50:57.875076 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:50:57.875076 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Jun 23 15:50:57.875076 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:50:57.875076 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:50:57.875076 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:50:57.875076 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:50:57.875076 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:50:57.875076 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:50:57.875076 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:50:57.875076 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:50:57.876916 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 15:50:57.876916 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 15:50:57.876916 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 15:50:57.876916 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Jun 23 15:50:57.876916 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 644/1251] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Jun 23 15:50:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 15:50:57.882297 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79f0d6b-0087-4836-9cba-ee833bc25b15 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:50:58.034018 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79f0d6b-0087-4836-9cba-ee833bc25b15 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:50:58.094179 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37002901-13c2-47d9-b6e4-d62ebd60fb74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:50:58.094788 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37002901-13c2-47d9-b6e4-d62ebd60fb74 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:50:58.095074 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37002901-13c2-47d9-b6e4-d62ebd60fb74 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:50:58.095354 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37002901-13c2-47d9-b6e4-d62ebd60fb74 None admin] PATH_INFO: `/v3/domains` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:50:58.096171 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37002901-13c2-47d9-b6e4-d62ebd60fb74 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:50:58.096419 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37002901-13c2-47d9-b6e4-d62ebd60fb74 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:50:58.097915 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37002901-13c2-47d9-b6e4-d62ebd60fb74 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:50:58.112157 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 608/1252] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Jun 23 15:50:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:50:58.117988 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983e6b9e-c8a2-452f-9486-8ea8684fbfbd None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:50:58.251635 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983e6b9e-c8a2-452f-9486-8ea8684fbfbd None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:50:58.298782 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:50:58.312543 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279d5bb4-4f41-4818-b0c3-1f4ea40835b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:50:58.313813 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279d5bb4-4f41-4818-b0c3-1f4ea40835b2 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:50:58.314128 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279d5bb4-4f41-4818-b0c3-1f4ea40835b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:50:58.316097 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279d5bb4-4f41-4818-b0c3-1f4ea40835b2 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:50:58.329309 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279d5bb4-4f41-4818-b0c3-1f4ea40835b2 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:50:58.331982 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279d5bb4-4f41-4818-b0c3-1f4ea40835b2 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:50:58.354932 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-279d5bb4-4f41-4818-b0c3-1f4ea40835b2 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 23 15:50:58.354932 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:50:58.354932 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:50:58.354932 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:50:58.354932 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:50:58.354932 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:50:58.354932 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:50:58.354932 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:50:58.354932 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:50:58.354932 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:50:58.354932 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:50:58.354932 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:50:58.354932 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 15:50:58.354932 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 15:50:58.354932 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 15:50:58.354932 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 15:50:58.356375 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:50:58.356375 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:50:58.356375 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 15:50:58.356375 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 15:50:58.356375 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 15:50:58.356375 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 15:50:58.356375 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:50:58.356375 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:50:58.356375 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:50:58.356375 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:50:58.356375 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:50:58.356375 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Jun 23 15:50:58.356375 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:50:58.356375 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:50:58.356375 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:50:58.356375 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:50:58.356375 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:50:58.356375 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:50:58.357484 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:50:58.357484 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:50:58.357484 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 15:50:58.357484 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 15:50:58.357484 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 15:50:58.357484 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 15:50:58.357484 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 15:50:58.357484 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 15:50:58.357484 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 23 15:50:58.357484 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Jun 23 15:50:58.361248 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 645/1253] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Jun 23 15:50:58 2022] GET /identity/v3/users/placement => generated 87 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:50:58.363216 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37002901-13c2-47d9-b6e4-d62ebd60fb74 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:50:58.483966 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:50:58.556745 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37002901-13c2-47d9-b6e4-d62ebd60fb74 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:50:58.607181 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17c9940-0c99-42f3-8be3-9d23064eecca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:50:58.607938 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17c9940-0c99-42f3-8be3-9d23064eecca None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:50:58.608229 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17c9940-0c99-42f3-8be3-9d23064eecca None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:50:58.608589 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17c9940-0c99-42f3-8be3-9d23064eecca None admin] PATH_INFO: `/v3/users` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:50:58.609602 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17c9940-0c99-42f3-8be3-9d23064eecca None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:50:58.609923 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17c9940-0c99-42f3-8be3-9d23064eecca None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:50:58.611768 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17c9940-0c99-42f3-8be3-9d23064eecca None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:50:58.636893 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 609/1254] 10.222.0.46 () {56 vars in 1143 bytes} [Thu Jun 23 15:50:58 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:50:58.644772 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279d5bb4-4f41-4818-b0c3-1f4ea40835b2 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:50:58.799603 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279d5bb4-4f41-4818-b0c3-1f4ea40835b2 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:50:58.855674 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba19ea8-92c9-4bba-b17d-4a168ed94240 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:50:58.857170 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba19ea8-92c9-4bba-b17d-4a168ed94240 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:50:58.857665 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba19ea8-92c9-4bba-b17d-4a168ed94240 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:50:58.858062 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba19ea8-92c9-4bba-b17d-4a168ed94240 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:50:58.947473 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:50:58.970193 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba19ea8-92c9-4bba-b17d-4a168ed94240 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:50:58.973090 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba19ea8-92c9-4bba-b17d-4a168ed94240 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:50:59.166952 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 646/1255] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Jun 23 15:50:58 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 518 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 23 15:50:59.169542 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17c9940-0c99-42f3-8be3-9d23064eecca None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:50:59.318427 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17c9940-0c99-42f3-8be3-9d23064eecca None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:50:59.337944 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:50:59.370403 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5912862-c6df-4d42-bda6-460f71646839 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:50:59.371280 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5912862-c6df-4d42-bda6-460f71646839 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:50:59.371565 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5912862-c6df-4d42-bda6-460f71646839 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:50:59.371842 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5912862-c6df-4d42-bda6-460f71646839 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:50:59.391219 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5912862-c6df-4d42-bda6-460f71646839 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:50:59.393897 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5912862-c6df-4d42-bda6-460f71646839 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:50:59.430585 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-e5912862-c6df-4d42-bda6-460f71646839 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 15:50:59.430585 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:50:59.430585 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:50:59.430585 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:50:59.430585 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:50:59.430585 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:50:59.430585 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:50:59.430585 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:50:59.430585 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:50:59.430585 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:50:59.430585 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:50:59.430585 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:50:59.430585 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 15:50:59.430585 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 15:50:59.430585 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 15:50:59.438063 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 15:50:59.438063 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:50:59.438063 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:50:59.438063 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:50:59.438063 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:50:59.438063 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:50:59.438063 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:50:59.438063 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:50:59.438063 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Jun 23 15:50:59.438063 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:50:59.438063 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:50:59.438063 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:50:59.438063 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:50:59.438063 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:50:59.438063 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:50:59.438063 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:50:59.438063 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:50:59.440066 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 15:50:59.440066 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 15:50:59.440066 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 15:50:59.440066 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Jun 23 15:50:59.440066 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 610/1256] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Jun 23 15:50:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:50:59.440622 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba19ea8-92c9-4bba-b17d-4a168ed94240 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:50:59.573165 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:50:59.596685 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba19ea8-92c9-4bba-b17d-4a168ed94240 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:50:59.659070 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d04fa81-4ae9-4ee5-a187-0477a80d528f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:50:59.659767 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d04fa81-4ae9-4ee5-a187-0477a80d528f None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:50:59.660070 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d04fa81-4ae9-4ee5-a187-0477a80d528f None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:50:59.660329 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d04fa81-4ae9-4ee5-a187-0477a80d528f None admin] PATH_INFO: `/v3/domains` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:50:59.661083 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d04fa81-4ae9-4ee5-a187-0477a80d528f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:50:59.661392 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d04fa81-4ae9-4ee5-a187-0477a80d528f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:50:59.662840 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d04fa81-4ae9-4ee5-a187-0477a80d528f None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:50:59.678954 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 647/1257] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Jun 23 15:50:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:50:59.685163 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5912862-c6df-4d42-bda6-460f71646839 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:50:59.830460 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5912862-c6df-4d42-bda6-460f71646839 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:50:59.880458 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718b10cf-b305-49dc-ba1e-04721821cf05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:50:59.881397 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718b10cf-b305-49dc-ba1e-04721821cf05 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:50:59.881759 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718b10cf-b305-49dc-ba1e-04721821cf05 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:50:59.882157 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718b10cf-b305-49dc-ba1e-04721821cf05 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:50:59.893604 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718b10cf-b305-49dc-ba1e-04721821cf05 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:50:59.895911 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718b10cf-b305-49dc-ba1e-04721821cf05 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:50:59.920420 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-718b10cf-b305-49dc-ba1e-04721821cf05 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 15:50:59.920420 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:50:59.920420 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:50:59.920420 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:50:59.920420 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:50:59.920420 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:50:59.920420 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:50:59.920420 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:50:59.920420 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:50:59.920420 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:50:59.920420 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:50:59.920420 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:50:59.920420 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 15:50:59.920420 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 15:50:59.920420 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 15:50:59.922218 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 15:50:59.922218 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:50:59.922218 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:50:59.922218 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:50:59.922218 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:50:59.922218 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:50:59.922218 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:50:59.922218 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:50:59.922218 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Jun 23 15:50:59.922218 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:50:59.922218 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:50:59.922218 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:50:59.922218 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:50:59.922218 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:50:59.922218 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:50:59.922218 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:50:59.922218 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:50:59.924372 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 15:50:59.924372 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 15:50:59.924372 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 15:50:59.924372 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 15:50:59.924372 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 15:50:59.924372 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 15:50:59.924372 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 15:50:59.924372 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Jun 23 15:50:59.924372 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 611/1258] 10.222.0.46 () {56 vars in 1111 bytes} [Thu Jun 23 15:50:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 15:50:59.929527 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d04fa81-4ae9-4ee5-a187-0477a80d528f None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:51:00.091044 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d04fa81-4ae9-4ee5-a187-0477a80d528f None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:51:00.142980 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4f904e-ba9e-42ea-bbca-7f14d475f593 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:51:00.143896 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4f904e-ba9e-42ea-bbca-7f14d475f593 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:51:00.144294 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4f904e-ba9e-42ea-bbca-7f14d475f593 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:51:00.144795 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4f904e-ba9e-42ea-bbca-7f14d475f593 None admin] PATH_INFO: `/v3/projects` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:51:00.145960 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4f904e-ba9e-42ea-bbca-7f14d475f593 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:51:00.146452 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4f904e-ba9e-42ea-bbca-7f14d475f593 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:51:00.148649 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4f904e-ba9e-42ea-bbca-7f14d475f593 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:51:00.176294 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 648/1259] 10.222.0.46 () {56 vars in 1148 bytes} [Thu Jun 23 15:50:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:51:00.177905 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718b10cf-b305-49dc-ba1e-04721821cf05 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:51:00.215569 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:51:00.337155 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718b10cf-b305-49dc-ba1e-04721821cf05 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:51:00.403946 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b5587a-fbbb-48bd-8603-e19648b98b41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:51:00.404994 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b5587a-fbbb-48bd-8603-e19648b98b41 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:51:00.405513 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b5587a-fbbb-48bd-8603-e19648b98b41 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:51:00.406030 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b5587a-fbbb-48bd-8603-e19648b98b41 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:51:00.407768 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b5587a-fbbb-48bd-8603-e19648b98b41 None admin] RBAC: Adding query filter params (role.id=812d3c567c52469cb2b35709fbdc857c, scope.project.id=28ecdf5b474c44098c722f677d2a0b77, user.id=3f66988b73644fe38b9849663ee70f25) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:51:00.408312 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b5587a-fbbb-48bd-8603-e19648b98b41 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:51:00.411216 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b5587a-fbbb-48bd-8603-e19648b98b41 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:51:00.420418 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 612/1260] 10.222.0.46 () {56 vars in 1374 bytes} [Thu Jun 23 15:51:00 2022] GET /identity/v3/role_assignments?user.id=3f66988b73644fe38b9849663ee70f25&scope.project.id=28ecdf5b474c44098c722f677d2a0b77&role.id=812d3c567c52469cb2b35709fbdc857c => generated 259 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:51:02.666657 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929da419-b4d5-41c6-b2c0-eef727c7bd60 None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:51:02.667248 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929da419-b4d5-41c6-b2c0-eef727c7bd60 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:51:02.667248 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929da419-b4d5-41c6-b2c0-eef727c7bd60 None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:51:02.668651 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 649/1261] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jun 23 15:51:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 15:51:02.674349 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1fd9af-ca6b-4293-be9e-6b8bae0f9882 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:51:02.674883 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1fd9af-ca6b-4293-be9e-6b8bae0f9882 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:51:02.675087 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1fd9af-ca6b-4293-be9e-6b8bae0f9882 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:51:02.693588 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-ba1fd9af-ca6b-4293-be9e-6b8bae0f9882 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:51:02.694940 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-ba1fd9af-ca6b-4293-be9e-6b8bae0f9882 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:51:02.772989 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 613/1262] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jun 23 15:51:02 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 15:51:02.797061 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929da419-b4d5-41c6-b2c0-eef727c7bd60 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:51:02.809073 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:51:02.941892 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929da419-b4d5-41c6-b2c0-eef727c7bd60 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:51:02.981795 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dea5d1c-7476-428b-a680-d09f3c40bea6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:51:02.982604 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dea5d1c-7476-428b-a680-d09f3c40bea6 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:51:02.982918 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dea5d1c-7476-428b-a680-d09f3c40bea6 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:51:02.983193 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dea5d1c-7476-428b-a680-d09f3c40bea6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:51:02.995071 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dea5d1c-7476-428b-a680-d09f3c40bea6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 15:51:03.003094 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-3dea5d1c-7476-428b-a680-d09f3c40bea6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 15:51:03.003094 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:51:03.003094 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:51:03.003094 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:51:03.003094 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:51:03.003094 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:51:03.003094 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:51:03.003094 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:51:03.003094 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:51:03.003094 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:51:03.003094 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:51:03.003094 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:51:03.003094 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 15:51:03.003094 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 15:51:03.003094 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 15:51:03.003094 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 15:51:03.004999 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 15:51:03.004999 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 15:51:03.004999 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 15:51:03.004999 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 15:51:03.004999 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:51:03.004999 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:51:03.004999 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:51:03.004999 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:51:03.004999 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:51:03.004999 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:51:03.004999 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:51:03.004999 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Jun 23 15:51:03.004999 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:51:03.004999 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:51:03.004999 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:51:03.004999 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:51:03.006180 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:51:03.006180 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:51:03.006180 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:51:03.006180 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:51:03.006180 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 15:51:03.006180 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 15:51:03.006180 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 15:51:03.006180 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 15:51:03.006180 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 15:51:03.006180 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 15:51:03.006180 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 15:51:03.006180 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Jun 23 15:51:03.006180 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 650/1263] 10.222.0.46 () {56 vars in 1096 bytes} [Thu Jun 23 15:51:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 15:51:03.010942 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1fd9af-ca6b-4293-be9e-6b8bae0f9882 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:51:03.032376 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:51:03.150336 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1fd9af-ca6b-4293-be9e-6b8bae0f9882 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:51:03.189883 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc995178-8530-4796-9b98-5f5b0309127d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:51:03.190577 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc995178-8530-4796-9b98-5f5b0309127d None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:51:03.190894 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc995178-8530-4796-9b98-5f5b0309127d None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:51:03.191176 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc995178-8530-4796-9b98-5f5b0309127d None admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:51:03.192006 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc995178-8530-4796-9b98-5f5b0309127d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:51:03.192267 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc995178-8530-4796-9b98-5f5b0309127d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:51:03.193673 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc995178-8530-4796-9b98-5f5b0309127d None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:51:03.203214 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 614/1264] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Jun 23 15:51:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:51:03.209996 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dea5d1c-7476-428b-a680-d09f3c40bea6 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:51:03.331563 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:51:03.375903 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dea5d1c-7476-428b-a680-d09f3c40bea6 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:51:03.414639 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d26f81-8a36-43db-87c6-52d1e33c9724 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:51:03.415605 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d26f81-8a36-43db-87c6-52d1e33c9724 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:51:03.415884 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d26f81-8a36-43db-87c6-52d1e33c9724 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:51:03.416157 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d26f81-8a36-43db-87c6-52d1e33c9724 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:51:03.429263 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d26f81-8a36-43db-87c6-52d1e33c9724 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:51:03.430781 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d26f81-8a36-43db-87c6-52d1e33c9724 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:51:03.449067 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-08d26f81-8a36-43db-87c6-52d1e33c9724 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 15:51:03.449067 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:51:03.449067 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:51:03.449067 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:51:03.449067 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:51:03.449067 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:51:03.449067 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:51:03.449067 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:51:03.449067 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:51:03.449067 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:51:03.449067 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:51:03.449067 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:51:03.449067 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 15:51:03.449067 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 15:51:03.449067 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 15:51:03.450763 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 15:51:03.450763 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:51:03.450763 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:51:03.450763 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:51:03.450763 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:51:03.450763 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:51:03.450763 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:51:03.450763 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:51:03.450763 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Jun 23 15:51:03.450763 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:51:03.450763 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:51:03.450763 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:51:03.450763 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:51:03.450763 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:51:03.450763 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:51:03.450763 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:51:03.450763 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:51:03.452653 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 15:51:03.452653 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 15:51:03.452653 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 15:51:03.452653 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Jun 23 15:51:03.452653 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 651/1265] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Jun 23 15:51:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:51:03.459187 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc995178-8530-4796-9b98-5f5b0309127d None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:51:03.553428 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:51:03.577160 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc995178-8530-4796-9b98-5f5b0309127d None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:51:03.614639 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a1f2b6-2aec-4b8f-97db-411d42a6e706 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:51:03.615358 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a1f2b6-2aec-4b8f-97db-411d42a6e706 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:51:03.615642 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a1f2b6-2aec-4b8f-97db-411d42a6e706 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:51:03.615916 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a1f2b6-2aec-4b8f-97db-411d42a6e706 None admin] PATH_INFO: `/v3/domains` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:51:03.616649 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a1f2b6-2aec-4b8f-97db-411d42a6e706 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:51:03.616956 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a1f2b6-2aec-4b8f-97db-411d42a6e706 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:51:03.618400 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a1f2b6-2aec-4b8f-97db-411d42a6e706 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:51:03.631872 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 615/1266] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Jun 23 15:51:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:51:03.641182 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d26f81-8a36-43db-87c6-52d1e33c9724 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:51:03.753762 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d26f81-8a36-43db-87c6-52d1e33c9724 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:51:03.777182 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:51:03.797247 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda873a6-470b-40f5-ae2d-ebe67deddaaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:51:03.798647 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda873a6-470b-40f5-ae2d-ebe67deddaaf None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:51:03.799207 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda873a6-470b-40f5-ae2d-ebe67deddaaf None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:51:03.799725 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda873a6-470b-40f5-ae2d-ebe67deddaaf None admin] PATH_INFO: `/v3/users/placement` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:51:03.814629 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda873a6-470b-40f5-ae2d-ebe67deddaaf None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:51:03.817373 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda873a6-470b-40f5-ae2d-ebe67deddaaf None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:51:03.846364 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-dda873a6-470b-40f5-ae2d-ebe67deddaaf None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 23 15:51:03.846364 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:51:03.846364 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:51:03.846364 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:51:03.846364 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:51:03.846364 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:51:03.846364 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:51:03.846364 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:51:03.846364 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:51:03.846364 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:51:03.846364 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:51:03.846364 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:51:03.846364 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 15:51:03.846364 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 15:51:03.846364 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 15:51:03.846364 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 15:51:03.847662 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:51:03.847662 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:51:03.847662 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 15:51:03.847662 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 15:51:03.847662 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 15:51:03.847662 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 15:51:03.847662 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:51:03.847662 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:51:03.847662 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:51:03.847662 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:51:03.847662 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:51:03.847662 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Jun 23 15:51:03.847662 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:51:03.847662 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:51:03.847662 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:51:03.847662 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:51:03.847662 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:51:03.847662 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:51:03.848771 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:51:03.848771 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:51:03.848771 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 15:51:03.848771 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 15:51:03.848771 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 15:51:03.848771 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 15:51:03.848771 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 15:51:03.848771 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 15:51:03.848771 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 23 15:51:03.848771 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Jun 23 15:51:03.848771 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 652/1267] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Jun 23 15:51:03 2022] GET /identity/v3/users/placement => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 15:51:03.857587 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a1f2b6-2aec-4b8f-97db-411d42a6e706 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:51:03.970064 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a1f2b6-2aec-4b8f-97db-411d42a6e706 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:51:04.008108 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906001dc-fe16-4c98-a8b8-b5b7a4416da5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:51:04.008714 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906001dc-fe16-4c98-a8b8-b5b7a4416da5 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:51:04.008997 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906001dc-fe16-4c98-a8b8-b5b7a4416da5 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:51:04.009272 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906001dc-fe16-4c98-a8b8-b5b7a4416da5 None admin] PATH_INFO: `/v3/users` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:51:04.010130 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906001dc-fe16-4c98-a8b8-b5b7a4416da5 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:51:04.010440 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906001dc-fe16-4c98-a8b8-b5b7a4416da5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:51:04.011939 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906001dc-fe16-4c98-a8b8-b5b7a4416da5 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:51:04.022947 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:51:04.033297 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 616/1268] 10.222.0.46 () {56 vars in 1143 bytes} [Thu Jun 23 15:51:03 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:51:04.039209 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda873a6-470b-40f5-ae2d-ebe67deddaaf None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:51:04.156902 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda873a6-470b-40f5-ae2d-ebe67deddaaf None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:51:04.197302 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccee87fa-7ada-4cb2-8cc9-074a11b0dcdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:51:04.198111 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccee87fa-7ada-4cb2-8cc9-074a11b0dcdc None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:51:04.198520 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccee87fa-7ada-4cb2-8cc9-074a11b0dcdc None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:51:04.198851 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccee87fa-7ada-4cb2-8cc9-074a11b0dcdc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:51:04.211246 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccee87fa-7ada-4cb2-8cc9-074a11b0dcdc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:51:04.212734 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccee87fa-7ada-4cb2-8cc9-074a11b0dcdc None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:51:04.232367 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-ccee87fa-7ada-4cb2-8cc9-074a11b0dcdc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 15:51:04.232367 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:51:04.232367 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:51:04.232367 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:51:04.232367 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:51:04.232367 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:51:04.232367 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:51:04.232367 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:51:04.232367 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:51:04.232367 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:51:04.232367 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:51:04.232367 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:51:04.232367 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 15:51:04.232367 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 15:51:04.232367 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 15:51:04.234075 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 15:51:04.234075 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:51:04.234075 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:51:04.234075 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:51:04.234075 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:51:04.234075 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:51:04.234075 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:51:04.234075 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:51:04.234075 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Jun 23 15:51:04.234075 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:51:04.234075 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:51:04.234075 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:51:04.234075 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:51:04.234075 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:51:04.234075 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:51:04.234075 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:51:04.234075 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:51:04.235072 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 15:51:04.235072 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 15:51:04.235072 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 15:51:04.235072 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Jun 23 15:51:04.235072 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 653/1269] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Jun 23 15:51:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:51:04.239865 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906001dc-fe16-4c98-a8b8-b5b7a4416da5 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:51:04.384000 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906001dc-fe16-4c98-a8b8-b5b7a4416da5 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:51:04.447535 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e973e0-b507-44cb-92fd-40983df82175 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:51:04.448727 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e973e0-b507-44cb-92fd-40983df82175 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:51:04.449252 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e973e0-b507-44cb-92fd-40983df82175 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:51:04.449741 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e973e0-b507-44cb-92fd-40983df82175 None admin] PATH_INFO: `/v3/domains` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:51:04.451169 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e973e0-b507-44cb-92fd-40983df82175 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:51:04.451697 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e973e0-b507-44cb-92fd-40983df82175 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:51:04.454362 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e973e0-b507-44cb-92fd-40983df82175 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:51:04.477062 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 617/1270] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Jun 23 15:51:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 15:51:04.483703 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccee87fa-7ada-4cb2-8cc9-074a11b0dcdc None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:51:04.521330 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:51:04.608267 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccee87fa-7ada-4cb2-8cc9-074a11b0dcdc None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:51:04.646394 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09381fda-77f2-4587-bd1a-ff352c220244 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:51:04.647490 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09381fda-77f2-4587-bd1a-ff352c220244 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:51:04.647809 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09381fda-77f2-4587-bd1a-ff352c220244 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:51:04.648118 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09381fda-77f2-4587-bd1a-ff352c220244 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:51:04.657530 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09381fda-77f2-4587-bd1a-ff352c220244 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:51:04.659090 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09381fda-77f2-4587-bd1a-ff352c220244 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:51:04.675569 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-09381fda-77f2-4587-bd1a-ff352c220244 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 15:51:04.675569 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:51:04.675569 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:51:04.675569 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:51:04.675569 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:51:04.675569 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:51:04.675569 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:51:04.675569 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:51:04.675569 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:51:04.675569 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:51:04.675569 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:51:04.675569 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:51:04.675569 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 15:51:04.675569 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 15:51:04.675569 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 15:51:04.677347 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 15:51:04.677347 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:51:04.677347 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:51:04.677347 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:51:04.677347 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:51:04.677347 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:51:04.677347 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:51:04.677347 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:51:04.677347 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Jun 23 15:51:04.677347 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:51:04.677347 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:51:04.677347 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:51:04.677347 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:51:04.677347 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:51:04.677347 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:51:04.677347 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:51:04.677347 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:51:04.679184 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 15:51:04.679184 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 15:51:04.679184 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 15:51:04.679184 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 15:51:04.679184 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 15:51:04.679184 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 15:51:04.679184 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 15:51:04.679184 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Jun 23 15:51:04.679184 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 654/1271] 10.222.0.46 () {56 vars in 1111 bytes} [Thu Jun 23 15:51:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 15:51:04.683181 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e973e0-b507-44cb-92fd-40983df82175 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:51:04.753902 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:51:04.803341 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e973e0-b507-44cb-92fd-40983df82175 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:51:04.869547 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05b767e-58c4-4bb4-a911-612247a71fd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:51:04.870754 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05b767e-58c4-4bb4-a911-612247a71fd6 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:51:04.871268 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05b767e-58c4-4bb4-a911-612247a71fd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:51:04.871797 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05b767e-58c4-4bb4-a911-612247a71fd6 None admin] PATH_INFO: `/v3/projects` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:51:04.873359 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05b767e-58c4-4bb4-a911-612247a71fd6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:51:04.873957 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05b767e-58c4-4bb4-a911-612247a71fd6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:51:04.876816 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05b767e-58c4-4bb4-a911-612247a71fd6 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:51:04.901508 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 618/1272] 10.222.0.46 () {56 vars in 1148 bytes} [Thu Jun 23 15:51:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:51:04.908328 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09381fda-77f2-4587-bd1a-ff352c220244 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:51:05.013211 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:51:05.023694 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09381fda-77f2-4587-bd1a-ff352c220244 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:51:05.063119 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fe2f97-e335-4179-b961-45dfd296954b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:51:05.064035 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fe2f97-e335-4179-b961-45dfd296954b None admin] REQUEST_METHOD: `PUT` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:51:05.064319 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fe2f97-e335-4179-b961-45dfd296954b None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:51:05.064596 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fe2f97-e335-4179-b961-45dfd296954b None admin] PATH_INFO: `/v3/projects/28ecdf5b474c44098c722f677d2a0b77/users/3f66988b73644fe38b9849663ee70f25/roles/812d3c567c52469cb2b35709fbdc857c` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:51:05.066940 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fe2f97-e335-4179-b961-45dfd296954b None admin] RBAC: Authorizing `identity:create_grant(project_id=28ecdf5b474c44098c722f677d2a0b77, user_id=3f66988b73644fe38b9849663ee70f25, role_id=812d3c567c52469cb2b35709fbdc857c)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:51:05.068423 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fe2f97-e335-4179-b961-45dfd296954b None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:51:05.084115 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 655/1273] 10.222.0.46 () {58 vars in 1439 bytes} [Thu Jun 23 15:51:04 2022] PUT /identity/v3/projects/28ecdf5b474c44098c722f677d2a0b77/users/3f66988b73644fe38b9849663ee70f25/roles/812d3c567c52469cb2b35709fbdc857c => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 15:51:08.382308 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14abf413-d957-497f-acae-f9c72a965fcb None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:51:08.383023 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14abf413-d957-497f-acae-f9c72a965fcb None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:51:08.383023 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14abf413-d957-497f-acae-f9c72a965fcb None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:51:08.384366 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 619/1274] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jun 23 15:51:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 15:51:08.390565 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76df0a9e-e87f-4968-a543-0ffa4c38c3f2 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:51:08.391174 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76df0a9e-e87f-4968-a543-0ffa4c38c3f2 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:51:08.391273 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76df0a9e-e87f-4968-a543-0ffa4c38c3f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:51:08.410759 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-76df0a9e-e87f-4968-a543-0ffa4c38c3f2 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:51:08.412150 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-76df0a9e-e87f-4968-a543-0ffa4c38c3f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:51:08.488884 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 656/1275] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jun 23 15:51:08 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 15:51:08.514042 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14abf413-d957-497f-acae-f9c72a965fcb None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:51:08.645269 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14abf413-d957-497f-acae-f9c72a965fcb None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:51:08.667100 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:51:08.690241 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a187637-dfbc-4fa0-9f84-90c8dda362ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:51:08.691144 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a187637-dfbc-4fa0-9f84-90c8dda362ca None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:51:08.691429 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a187637-dfbc-4fa0-9f84-90c8dda362ca None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:51:08.691705 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a187637-dfbc-4fa0-9f84-90c8dda362ca None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:51:08.705122 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a187637-dfbc-4fa0-9f84-90c8dda362ca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 23 15:51:08.713147 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-2a187637-dfbc-4fa0-9f84-90c8dda362ca None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 15:51:08.713147 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:51:08.713147 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:51:08.713147 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:51:08.713147 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:51:08.713147 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:51:08.713147 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:51:08.713147 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:51:08.713147 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:51:08.713147 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:51:08.713147 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:51:08.713147 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:51:08.713147 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 23 15:51:08.713147 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 23 15:51:08.713147 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 23 15:51:08.713147 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 23 15:51:08.715088 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 23 15:51:08.715088 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 23 15:51:08.715088 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 23 15:51:08.715088 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 23 15:51:08.715088 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:51:08.715088 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:51:08.715088 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:51:08.715088 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:51:08.715088 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:51:08.715088 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:51:08.715088 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:51:08.715088 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Jun 23 15:51:08.715088 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:51:08.715088 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:51:08.715088 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:51:08.715088 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:51:08.717357 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:51:08.717357 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:51:08.717357 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:51:08.717357 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:51:08.717357 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 23 15:51:08.717357 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 23 15:51:08.717357 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 23 15:51:08.717357 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 23 15:51:08.717357 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 23 15:51:08.717357 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 23 15:51:08.717357 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 23 15:51:08.717357 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Jun 23 15:51:08.717357 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 620/1276] 10.222.0.46 () {56 vars in 1096 bytes} [Thu Jun 23 15:51:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:51:08.723389 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76df0a9e-e87f-4968-a543-0ffa4c38c3f2 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:51:08.825657 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:51:08.849249 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76df0a9e-e87f-4968-a543-0ffa4c38c3f2 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:51:08.889612 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3a54ca-9e72-4fd6-a356-401c4a3d2b5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:51:08.890336 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3a54ca-9e72-4fd6-a356-401c4a3d2b5d None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:51:08.890625 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3a54ca-9e72-4fd6-a356-401c4a3d2b5d None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:51:08.890933 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3a54ca-9e72-4fd6-a356-401c4a3d2b5d None admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:51:08.891762 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3a54ca-9e72-4fd6-a356-401c4a3d2b5d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:51:08.892073 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3a54ca-9e72-4fd6-a356-401c4a3d2b5d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:51:08.893487 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3a54ca-9e72-4fd6-a356-401c4a3d2b5d None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:51:08.903410 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 657/1277] 10.222.0.46 () {56 vars in 1099 bytes} [Thu Jun 23 15:51:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:51:08.911896 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a187637-dfbc-4fa0-9f84-90c8dda362ca None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:51:09.031318 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a187637-dfbc-4fa0-9f84-90c8dda362ca None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:51:09.069964 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f7a4f1-8fd7-457d-9926-a14b040c73b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:51:09.070810 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f7a4f1-8fd7-457d-9926-a14b040c73b5 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:51:09.071101 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f7a4f1-8fd7-457d-9926-a14b040c73b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:51:09.071382 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f7a4f1-8fd7-457d-9926-a14b040c73b5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:51:09.084191 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f7a4f1-8fd7-457d-9926-a14b040c73b5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:51:09.085690 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f7a4f1-8fd7-457d-9926-a14b040c73b5 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:51:09.105131 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-59f7a4f1-8fd7-457d-9926-a14b040c73b5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 15:51:09.105131 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:51:09.105131 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:51:09.105131 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:51:09.105131 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:51:09.105131 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:51:09.105131 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:51:09.105131 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:51:09.105131 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:51:09.105131 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:51:09.105131 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:51:09.105131 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:51:09.105131 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 15:51:09.105131 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 15:51:09.105131 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 15:51:09.106878 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 15:51:09.106878 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:51:09.106878 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:51:09.106878 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:51:09.106878 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:51:09.106878 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:51:09.106878 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:51:09.106878 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:51:09.106878 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Jun 23 15:51:09.106878 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:51:09.106878 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:51:09.106878 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:51:09.106878 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:51:09.106878 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:51:09.106878 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:51:09.106878 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:51:09.106878 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:51:09.108766 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 15:51:09.108766 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 15:51:09.108766 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 15:51:09.108766 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Jun 23 15:51:09.108766 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 621/1278] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Jun 23 15:51:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:51:09.113899 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3a54ca-9e72-4fd6-a356-401c4a3d2b5d None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:51:09.259925 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3a54ca-9e72-4fd6-a356-401c4a3d2b5d None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:51:09.309899 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c8732f-76d4-4bdb-aa78-7d136b0db556 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:51:09.311099 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c8732f-76d4-4bdb-aa78-7d136b0db556 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:51:09.311557 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c8732f-76d4-4bdb-aa78-7d136b0db556 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:51:09.311952 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c8732f-76d4-4bdb-aa78-7d136b0db556 None admin] PATH_INFO: `/v3/domains` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:51:09.313238 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c8732f-76d4-4bdb-aa78-7d136b0db556 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:51:09.313885 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c8732f-76d4-4bdb-aa78-7d136b0db556 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:51:09.316151 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c8732f-76d4-4bdb-aa78-7d136b0db556 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:51:09.335273 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 658/1279] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Jun 23 15:51:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:51:09.343957 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f7a4f1-8fd7-457d-9926-a14b040c73b5 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:51:09.387711 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:51:09.480698 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f7a4f1-8fd7-457d-9926-a14b040c73b5 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:51:09.518733 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0066b58a-fa23-41b1-a279-3ad12778d3c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:51:09.519480 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0066b58a-fa23-41b1-a279-3ad12778d3c7 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:51:09.519769 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0066b58a-fa23-41b1-a279-3ad12778d3c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:51:09.520048 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0066b58a-fa23-41b1-a279-3ad12778d3c7 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:51:09.529022 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0066b58a-fa23-41b1-a279-3ad12778d3c7 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:51:09.530514 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0066b58a-fa23-41b1-a279-3ad12778d3c7 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:51:09.546833 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-0066b58a-fa23-41b1-a279-3ad12778d3c7 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 23 15:51:09.546833 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:51:09.546833 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:51:09.546833 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:51:09.546833 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:51:09.546833 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:51:09.546833 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:51:09.546833 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:51:09.546833 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:51:09.546833 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:51:09.546833 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:51:09.546833 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:51:09.546833 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 23 15:51:09.546833 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 23 15:51:09.546833 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 23 15:51:09.546833 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 23 15:51:09.548679 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:51:09.548679 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:51:09.548679 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 23 15:51:09.548679 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 15:51:09.548679 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 23 15:51:09.548679 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 23 15:51:09.548679 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:51:09.548679 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:51:09.548679 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:51:09.548679 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:51:09.548679 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:51:09.548679 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Jun 23 15:51:09.548679 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:51:09.548679 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:51:09.548679 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:51:09.548679 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:51:09.548679 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:51:09.548679 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:51:09.549973 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:51:09.549973 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:51:09.549973 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 23 15:51:09.549973 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 23 15:51:09.549973 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 23 15:51:09.549973 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 23 15:51:09.549973 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 23 15:51:09.549973 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 23 15:51:09.549973 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 23 15:51:09.549973 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Jun 23 15:51:09.549973 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 622/1280] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Jun 23 15:51:09 2022] GET /identity/v3/users/placement => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:51:09.554676 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c8732f-76d4-4bdb-aa78-7d136b0db556 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:51:09.560729 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:51:09.733874 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c8732f-76d4-4bdb-aa78-7d136b0db556 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:51:09.799027 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4b6f82-dec0-47f3-9fb0-09fe1b9667e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:51:09.800108 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4b6f82-dec0-47f3-9fb0-09fe1b9667e2 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:51:09.800616 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4b6f82-dec0-47f3-9fb0-09fe1b9667e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:51:09.801105 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4b6f82-dec0-47f3-9fb0-09fe1b9667e2 None admin] PATH_INFO: `/v3/users` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:51:09.802860 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4b6f82-dec0-47f3-9fb0-09fe1b9667e2 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:51:09.803438 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4b6f82-dec0-47f3-9fb0-09fe1b9667e2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:51:09.806205 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4b6f82-dec0-47f3-9fb0-09fe1b9667e2 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:51:09.833386 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 659/1281] 10.222.0.46 () {56 vars in 1143 bytes} [Thu Jun 23 15:51:09 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:51:09.840037 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0066b58a-fa23-41b1-a279-3ad12778d3c7 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:51:09.946761 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:51:09.959100 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0066b58a-fa23-41b1-a279-3ad12778d3c7 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:51:09.998672 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f48fcb-e10b-4b81-a884-2c8395d356e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:51:09.999384 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f48fcb-e10b-4b81-a884-2c8395d356e5 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:51:09.999669 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f48fcb-e10b-4b81-a884-2c8395d356e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:51:09.999950 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f48fcb-e10b-4b81-a884-2c8395d356e5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:51:10.080392 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f48fcb-e10b-4b81-a884-2c8395d356e5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:51:10.081830 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f48fcb-e10b-4b81-a884-2c8395d356e5 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:51:10.191210 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:51:10.205014 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 623/1282] 10.222.0.46 () {58 vars in 1282 bytes} [Thu Jun 23 15:51:09 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 23 15:51:10.214740 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4b6f82-dec0-47f3-9fb0-09fe1b9667e2 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:51:10.307890 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:51:10.356069 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4b6f82-dec0-47f3-9fb0-09fe1b9667e2 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:51:10.397601 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7728c03-3dfb-4c42-864b-8a2e35919a75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:51:10.398507 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7728c03-3dfb-4c42-864b-8a2e35919a75 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:51:10.398945 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7728c03-3dfb-4c42-864b-8a2e35919a75 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:51:10.399307 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7728c03-3dfb-4c42-864b-8a2e35919a75 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:51:10.411083 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7728c03-3dfb-4c42-864b-8a2e35919a75 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:51:10.412643 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7728c03-3dfb-4c42-864b-8a2e35919a75 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:51:10.432907 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-a7728c03-3dfb-4c42-864b-8a2e35919a75 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 15:51:10.432907 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:51:10.432907 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:51:10.432907 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:51:10.432907 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:51:10.432907 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:51:10.432907 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:51:10.432907 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:51:10.432907 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:51:10.432907 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:51:10.432907 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:51:10.432907 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:51:10.432907 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 23 15:51:10.432907 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 23 15:51:10.432907 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 23 15:51:10.435164 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 23 15:51:10.435164 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:51:10.435164 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:51:10.435164 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:51:10.435164 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:51:10.435164 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:51:10.435164 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:51:10.435164 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:51:10.435164 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Jun 23 15:51:10.435164 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:51:10.435164 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:51:10.435164 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:51:10.435164 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:51:10.435164 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:51:10.435164 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:51:10.435164 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:51:10.435164 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:51:10.436466 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 23 15:51:10.436466 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 23 15:51:10.436466 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 23 15:51:10.436466 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Jun 23 15:51:10.436466 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 660/1283] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Jun 23 15:51:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:51:10.442062 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f48fcb-e10b-4b81-a884-2c8395d356e5 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:51:10.558005 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f48fcb-e10b-4b81-a884-2c8395d356e5 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:51:10.608794 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f48dd5b-8457-4318-b41a-b75e9ec5d418 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:51:10.609922 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f48dd5b-8457-4318-b41a-b75e9ec5d418 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:51:10.610430 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f48dd5b-8457-4318-b41a-b75e9ec5d418 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:51:10.611004 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f48dd5b-8457-4318-b41a-b75e9ec5d418 None admin] PATH_INFO: `/v3/domains` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:51:10.612491 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f48dd5b-8457-4318-b41a-b75e9ec5d418 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:51:10.613068 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f48dd5b-8457-4318-b41a-b75e9ec5d418 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:51:10.615252 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f48dd5b-8457-4318-b41a-b75e9ec5d418 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:51:10.628951 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 624/1284] 10.222.0.46 () {56 vars in 1109 bytes} [Thu Jun 23 15:51:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 15:51:10.635094 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7728c03-3dfb-4c42-864b-8a2e35919a75 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:51:10.756903 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7728c03-3dfb-4c42-864b-8a2e35919a75 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:51:10.774165 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:51:10.800956 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4d9bca-e0eb-46ee-aeb0-9b6154002bc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:51:10.801782 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4d9bca-e0eb-46ee-aeb0-9b6154002bc0 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:51:10.802067 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4d9bca-e0eb-46ee-aeb0-9b6154002bc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:51:10.802345 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4d9bca-e0eb-46ee-aeb0-9b6154002bc0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:51:10.811245 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4d9bca-e0eb-46ee-aeb0-9b6154002bc0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:51:10.812836 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4d9bca-e0eb-46ee-aeb0-9b6154002bc0 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:51:10.829555 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application [None req-1b4d9bca-e0eb-46ee-aeb0-9b6154002bc0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 15:51:10.829555 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:51:10.829555 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:51:10.829555 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:51:10.829555 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:51:10.829555 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:51:10.829555 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:51:10.829555 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:51:10.829555 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:51:10.829555 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:51:10.829555 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:51:10.829555 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:51:10.829555 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 23 15:51:10.829555 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 23 15:51:10.829555 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 23 15:51:10.835256 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 23 15:51:10.835256 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:51:10.835256 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:51:10.835256 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:51:10.835256 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:51:10.835256 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:51:10.835256 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:51:10.835256 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:51:10.835256 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application with Lock( Jun 23 15:51:10.835256 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:51:10.835256 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:51:10.835256 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:51:10.835256 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:51:10.835256 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:51:10.835256 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:51:10.835256 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:51:10.835256 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:51:10.837148 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 23 15:51:10.837148 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 23 15:51:10.837148 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 23 15:51:10.837148 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 23 15:51:10.837148 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 23 15:51:10.837148 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 23 15:51:10.837148 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 23 15:51:10.837148 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: ERROR keystone.server.flask.application Jun 23 15:51:10.837148 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 661/1285] 10.222.0.46 () {56 vars in 1111 bytes} [Thu Jun 23 15:51:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 15:51:10.839141 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f48dd5b-8457-4318-b41a-b75e9ec5d418 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:51:10.901474 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:51:10.961900 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f48dd5b-8457-4318-b41a-b75e9ec5d418 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:51:11.000166 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea46cd9-325b-4d6f-9871-9729ef81a225 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:51:11.000916 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea46cd9-325b-4d6f-9871-9729ef81a225 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:51:11.001249 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea46cd9-325b-4d6f-9871-9729ef81a225 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:51:11.001578 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea46cd9-325b-4d6f-9871-9729ef81a225 None admin] PATH_INFO: `/v3/projects` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:51:11.002443 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea46cd9-325b-4d6f-9871-9729ef81a225 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:51:11.002855 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea46cd9-325b-4d6f-9871-9729ef81a225 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:51:11.004368 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea46cd9-325b-4d6f-9871-9729ef81a225 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:51:11.018173 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 625/1286] 10.222.0.46 () {56 vars in 1148 bytes} [Thu Jun 23 15:51:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:51:11.024857 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4d9bca-e0eb-46ee-aeb0-9b6154002bc0 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:51:11.140847 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4d9bca-e0eb-46ee-aeb0-9b6154002bc0 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:51:11.182808 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b588c1-9e51-4dd8-99bd-47f3a6187c3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:51:11.183434 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b588c1-9e51-4dd8-99bd-47f3a6187c3f None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:51:11.183724 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b588c1-9e51-4dd8-99bd-47f3a6187c3f None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:51:11.184030 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b588c1-9e51-4dd8-99bd-47f3a6187c3f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:51:11.184904 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b588c1-9e51-4dd8-99bd-47f3a6187c3f None admin] RBAC: Adding query filter params (role.id=812d3c567c52469cb2b35709fbdc857c, scope.project.id=28ecdf5b474c44098c722f677d2a0b77, user.id=3f66988b73644fe38b9849663ee70f25) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:51:11.185221 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b588c1-9e51-4dd8-99bd-47f3a6187c3f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:51:11.187027 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b588c1-9e51-4dd8-99bd-47f3a6187c3f None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:51:11.194179 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 662/1287] 10.222.0.46 () {56 vars in 1374 bytes} [Thu Jun 23 15:51:11 2022] GET /identity/v3/role_assignments?user.id=3f66988b73644fe38b9849663ee70f25&scope.project.id=28ecdf5b474c44098c722f677d2a0b77&role.id=812d3c567c52469cb2b35709fbdc857c => generated 608 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:51:13.474500 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdfece1-4475-4c54-9134-5d942cf4564f None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:51:13.475091 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdfece1-4475-4c54-9134-5d942cf4564f None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:51:13.475293 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdfece1-4475-4c54-9134-5d942cf4564f None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:51:13.476633 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 626/1288] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jun 23 15:51:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 15:51:13.483202 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be93558-38b5-43cd-a5b1-1e92e49daf2a None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:51:13.483793 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be93558-38b5-43cd-a5b1-1e92e49daf2a None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:51:13.483793 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be93558-38b5-43cd-a5b1-1e92e49daf2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:51:13.502020 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-8be93558-38b5-43cd-a5b1-1e92e49daf2a None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:51:13.503452 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-8be93558-38b5-43cd-a5b1-1e92e49daf2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:51:13.540160 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:51:13.593765 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 663/1289] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jun 23 15:51:13 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 15:51:13.617751 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdfece1-4475-4c54-9134-5d942cf4564f None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:51:13.738204 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdfece1-4475-4c54-9134-5d942cf4564f None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:51:13.751422 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:51:13.779534 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c3f3ec-8756-4a58-ae06-b1dda990aace None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:51:13.780371 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c3f3ec-8756-4a58-ae06-b1dda990aace None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:51:13.780659 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c3f3ec-8756-4a58-ae06-b1dda990aace None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:51:13.780937 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c3f3ec-8756-4a58-ae06-b1dda990aace None admin] PATH_INFO: `/v3/services/placement` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:51:13.781617 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c3f3ec-8756-4a58-ae06-b1dda990aace None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:51:13.783088 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c3f3ec-8756-4a58-ae06-b1dda990aace None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:51:13.796831 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-d2c3f3ec-8756-4a58-ae06-b1dda990aace None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 23 15:51:13.796831 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:51:13.796831 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 15:51:13.796831 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 15:51:13.796831 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 15:51:13.796831 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 15:51:13.796831 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 15:51:13.796831 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 15:51:13.796831 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 23 15:51:13.796831 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Jun 23 15:51:13.796831 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 15:51:13.796831 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Jun 23 15:51:13.796831 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:51:13.796831 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:51:13.796831 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:51:13.796831 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:51:13.796831 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:51:13.798811 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:51:13.798811 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:51:13.798811 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:51:13.798811 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:51:13.798811 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:51:13.798811 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:51:13.798811 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 15:51:13.798811 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 15:51:13.798811 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 15:51:13.798811 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 15:51:13.798811 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:51:13.798811 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:51:13.798811 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:51:13.798811 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:51:13.798811 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:51:13.798811 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:51:13.800481 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:51:13.800481 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Jun 23 15:51:13.800481 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:51:13.800481 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:51:13.800481 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:51:13.800481 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:51:13.800481 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:51:13.800481 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:51:13.800481 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:51:13.800481 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:51:13.800481 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 15:51:13.800481 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 15:51:13.800481 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 23 15:51:13.800481 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Jun 23 15:51:13.800481 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 627/1290] 10.222.0.46 () {56 vars in 1117 bytes} [Thu Jun 23 15:51:13 2022] GET /identity/v3/services/placement => generated 90 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 23 15:51:13.806746 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be93558-38b5-43cd-a5b1-1e92e49daf2a None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:51:13.923440 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be93558-38b5-43cd-a5b1-1e92e49daf2a None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:51:13.962993 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47742ae6-0b09-4a3b-ba17-1c0403400965 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:51:13.963672 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47742ae6-0b09-4a3b-ba17-1c0403400965 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:51:13.963956 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47742ae6-0b09-4a3b-ba17-1c0403400965 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:51:13.964232 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47742ae6-0b09-4a3b-ba17-1c0403400965 None admin] PATH_INFO: `/v3/services` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:51:13.964988 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47742ae6-0b09-4a3b-ba17-1c0403400965 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:51:13.965305 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47742ae6-0b09-4a3b-ba17-1c0403400965 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:51:13.966765 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47742ae6-0b09-4a3b-ba17-1c0403400965 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:51:13.972507 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 664/1291] 10.222.0.46 () {56 vars in 1116 bytes} [Thu Jun 23 15:51:13 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:51:13.982040 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c3f3ec-8756-4a58-ae06-b1dda990aace None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:51:14.099561 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c3f3ec-8756-4a58-ae06-b1dda990aace None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:51:14.140407 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee200c1-1333-4502-a456-9ae1b4592e27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:51:14.141185 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee200c1-1333-4502-a456-9ae1b4592e27 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:51:14.141536 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee200c1-1333-4502-a456-9ae1b4592e27 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:51:14.141868 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee200c1-1333-4502-a456-9ae1b4592e27 None admin] PATH_INFO: `/v3/services` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:51:14.142729 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee200c1-1333-4502-a456-9ae1b4592e27 None admin] RBAC: Adding query filter params (type=placement) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:51:14.143259 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee200c1-1333-4502-a456-9ae1b4592e27 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:51:14.144784 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee200c1-1333-4502-a456-9ae1b4592e27 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:51:14.150598 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 628/1292] 10.222.0.46 () {56 vars in 1116 bytes} [Thu Jun 23 15:51:13 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:51:16.309048 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d512c8-a857-4d24-91da-ab7f12dc529d None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:51:16.309667 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d512c8-a857-4d24-91da-ab7f12dc529d None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:51:16.309667 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d512c8-a857-4d24-91da-ab7f12dc529d None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:51:16.311931 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 665/1293] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jun 23 15:51:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 15:51:16.318097 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73002e5-c6bf-4dbf-b11e-649fb45c1dc4 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:51:16.318654 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73002e5-c6bf-4dbf-b11e-649fb45c1dc4 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:51:16.318817 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73002e5-c6bf-4dbf-b11e-649fb45c1dc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:51:16.337095 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-f73002e5-c6bf-4dbf-b11e-649fb45c1dc4 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:51:16.338378 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-f73002e5-c6bf-4dbf-b11e-649fb45c1dc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:51:16.373491 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:51:16.417133 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 629/1294] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jun 23 15:51:16 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 15:51:16.440883 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d512c8-a857-4d24-91da-ab7f12dc529d None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:51:16.476775 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:51:16.560525 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d512c8-a857-4d24-91da-ab7f12dc529d None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:51:16.601225 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265c185a-02c8-4c00-9260-0a18e7515800 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:51:16.601904 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265c185a-02c8-4c00-9260-0a18e7515800 None admin] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:51:16.602191 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265c185a-02c8-4c00-9260-0a18e7515800 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:51:16.602470 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265c185a-02c8-4c00-9260-0a18e7515800 None admin] PATH_INFO: `/v3/services` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:51:16.603265 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265c185a-02c8-4c00-9260-0a18e7515800 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:51:16.604686 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265c185a-02c8-4c00-9260-0a18e7515800 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:51:16.620364 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 666/1295] 10.222.0.46 () {60 vars in 1141 bytes} [Thu Jun 23 15:51:16 2022] POST /identity/v3/services => generated 246 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 15:51:18.963435 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1b5c38-087d-4281-90de-fb6b543249c8 None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:51:18.964303 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1b5c38-087d-4281-90de-fb6b543249c8 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:51:18.964303 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1b5c38-087d-4281-90de-fb6b543249c8 None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:51:18.965464 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 630/1296] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jun 23 15:51:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 15:51:18.971639 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0b70cd-d7cc-4b17-aabe-bcc9be0a3b25 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:51:18.972240 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0b70cd-d7cc-4b17-aabe-bcc9be0a3b25 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:51:18.972240 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0b70cd-d7cc-4b17-aabe-bcc9be0a3b25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:51:18.991392 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-3f0b70cd-d7cc-4b17-aabe-bcc9be0a3b25 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:51:18.992678 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-3f0b70cd-d7cc-4b17-aabe-bcc9be0a3b25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:51:19.082971 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 667/1297] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jun 23 15:51:18 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 15:51:19.107336 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1b5c38-087d-4281-90de-fb6b543249c8 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:51:19.228840 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1b5c38-087d-4281-90de-fb6b543249c8 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:51:19.269603 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1caddd08-954f-4d54-ac96-74f01e7d0c3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:51:19.270437 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1caddd08-954f-4d54-ac96-74f01e7d0c3b None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:51:19.270903 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1caddd08-954f-4d54-ac96-74f01e7d0c3b None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:51:19.271042 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1caddd08-954f-4d54-ac96-74f01e7d0c3b None admin] PATH_INFO: `/v3/services/placement` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:51:19.271962 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1caddd08-954f-4d54-ac96-74f01e7d0c3b None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:51:19.273375 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1caddd08-954f-4d54-ac96-74f01e7d0c3b None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:51:19.286484 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-1caddd08-954f-4d54-ac96-74f01e7d0c3b None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 23 15:51:19.286484 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:51:19.286484 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 15:51:19.286484 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 15:51:19.286484 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 15:51:19.286484 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 15:51:19.286484 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 15:51:19.286484 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 15:51:19.286484 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 23 15:51:19.286484 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Jun 23 15:51:19.286484 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 15:51:19.286484 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Jun 23 15:51:19.286484 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:51:19.286484 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:51:19.286484 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:51:19.286484 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:51:19.286484 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:51:19.288275 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:51:19.288275 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:51:19.288275 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:51:19.288275 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:51:19.288275 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:51:19.288275 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:51:19.288275 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 15:51:19.288275 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 15:51:19.288275 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 15:51:19.288275 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 15:51:19.288275 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:51:19.288275 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:51:19.288275 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:51:19.288275 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:51:19.288275 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:51:19.288275 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:51:19.289152 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:51:19.289152 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Jun 23 15:51:19.289152 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:51:19.289152 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:51:19.289152 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:51:19.289152 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:51:19.289152 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:51:19.289152 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:51:19.289152 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:51:19.289152 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:51:19.289152 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 15:51:19.289152 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 15:51:19.289152 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 23 15:51:19.289152 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Jun 23 15:51:19.289152 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 631/1298] 10.222.0.46 () {56 vars in 1117 bytes} [Thu Jun 23 15:51:19 2022] GET /identity/v3/services/placement => generated 90 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:51:19.297793 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0b70cd-d7cc-4b17-aabe-bcc9be0a3b25 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:51:19.318821 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:51:19.416284 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0b70cd-d7cc-4b17-aabe-bcc9be0a3b25 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:51:19.456213 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c56c0d-9410-4b7c-a915-7235c0dd5c82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:51:19.456897 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c56c0d-9410-4b7c-a915-7235c0dd5c82 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:51:19.457182 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c56c0d-9410-4b7c-a915-7235c0dd5c82 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:51:19.457461 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c56c0d-9410-4b7c-a915-7235c0dd5c82 None admin] PATH_INFO: `/v3/services` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:51:19.458185 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c56c0d-9410-4b7c-a915-7235c0dd5c82 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:51:19.458495 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c56c0d-9410-4b7c-a915-7235c0dd5c82 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:51:19.459949 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c56c0d-9410-4b7c-a915-7235c0dd5c82 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:51:19.466657 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 668/1299] 10.222.0.46 () {56 vars in 1116 bytes} [Thu Jun 23 15:51:19 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:51:19.476038 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1caddd08-954f-4d54-ac96-74f01e7d0c3b None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:51:19.513207 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:51:19.595975 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1caddd08-954f-4d54-ac96-74f01e7d0c3b None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:51:19.635540 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6496088b-3da6-446e-b619-8a7230b32392 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:51:19.636229 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6496088b-3da6-446e-b619-8a7230b32392 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:51:19.636515 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6496088b-3da6-446e-b619-8a7230b32392 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:51:19.636847 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6496088b-3da6-446e-b619-8a7230b32392 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:51:19.637665 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6496088b-3da6-446e-b619-8a7230b32392 None admin] RBAC: Adding query filter params (interface=public, service_id=377bacc4bf4d4bb9a5dad26310342ed4, region_id=RegionOne) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:51:19.637983 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6496088b-3da6-446e-b619-8a7230b32392 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:51:19.639432 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6496088b-3da6-446e-b619-8a7230b32392 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:51:19.645575 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 632/1300] 10.222.0.46 () {56 vars in 1251 bytes} [Thu Jun 23 15:51:19 2022] GET /identity/v3/endpoints?service_id=377bacc4bf4d4bb9a5dad26310342ed4&interface=public®ion_id=RegionOne => generated 194 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:51:19.655239 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c56c0d-9410-4b7c-a915-7235c0dd5c82 None admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:51:19.751218 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:51:19.773538 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c56c0d-9410-4b7c-a915-7235c0dd5c82 None admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:51:19.812734 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519aeff3-b4ef-4813-8d35-9de55e19ca84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:51:19.813426 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519aeff3-b4ef-4813-8d35-9de55e19ca84 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:51:19.813712 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519aeff3-b4ef-4813-8d35-9de55e19ca84 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:51:19.813988 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519aeff3-b4ef-4813-8d35-9de55e19ca84 None admin] PATH_INFO: `/v3/services` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:51:19.814715 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519aeff3-b4ef-4813-8d35-9de55e19ca84 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:51:19.816136 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519aeff3-b4ef-4813-8d35-9de55e19ca84 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:51:19.823881 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 669/1301] 10.222.0.46 () {56 vars in 1087 bytes} [Thu Jun 23 15:51:19 2022] GET /identity/v3/services => generated 2191 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 15:51:21.981476 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7410c7-07e9-40a8-b388-273a6143f121 None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:51:21.982048 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7410c7-07e9-40a8-b388-273a6143f121 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:51:21.982048 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7410c7-07e9-40a8-b388-273a6143f121 None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:51:21.983657 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 633/1302] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jun 23 15:51:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 15:51:21.989623 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319c7d37-8e37-4fe8-bc95-40d51439d28d None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:51:21.990163 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319c7d37-8e37-4fe8-bc95-40d51439d28d None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:51:21.990163 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319c7d37-8e37-4fe8-bc95-40d51439d28d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:51:22.008746 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-319c7d37-8e37-4fe8-bc95-40d51439d28d None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:51:22.009990 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-319c7d37-8e37-4fe8-bc95-40d51439d28d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:51:22.087774 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 670/1303] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jun 23 15:51:21 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 23 15:51:22.113148 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7410c7-07e9-40a8-b388-273a6143f121 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:51:22.238576 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:51:22.246088 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7410c7-07e9-40a8-b388-273a6143f121 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:51:22.287836 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a56981-5e00-4431-b3bb-da1bb282fe83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:51:22.288667 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a56981-5e00-4431-b3bb-da1bb282fe83 None admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:51:22.288952 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a56981-5e00-4431-b3bb-da1bb282fe83 None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:51:22.289233 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a56981-5e00-4431-b3bb-da1bb282fe83 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:51:22.289905 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a56981-5e00-4431-b3bb-da1bb282fe83 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:51:22.291384 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a56981-5e00-4431-b3bb-da1bb282fe83 None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:51:22.304536 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application [None req-59a56981-5e00-4431-b3bb-da1bb282fe83 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 23 15:51:22.304536 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:51:22.304536 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 23 15:51:22.304536 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 23 15:51:22.304536 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 23 15:51:22.304536 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 23 15:51:22.304536 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 23 15:51:22.304536 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 15:51:22.304536 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 23 15:51:22.304536 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Jun 23 15:51:22.304536 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 23 15:51:22.304536 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Jun 23 15:51:22.304536 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 23 15:51:22.304536 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 23 15:51:22.304536 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 23 15:51:22.304536 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 23 15:51:22.304536 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 23 15:51:22.306439 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 23 15:51:22.306439 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 23 15:51:22.306439 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 23 15:51:22.306439 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 23 15:51:22.306439 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 23 15:51:22.306439 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 23 15:51:22.306439 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 23 15:51:22.306439 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 23 15:51:22.306439 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 23 15:51:22.306439 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 23 15:51:22.306439 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 23 15:51:22.306439 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 23 15:51:22.306439 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 23 15:51:22.306439 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 23 15:51:22.306439 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 23 15:51:22.306439 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.get_or_create( Jun 23 15:51:22.308441 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 23 15:51:22.308441 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application with Lock( Jun 23 15:51:22.308441 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 23 15:51:22.308441 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self._enter() Jun 23 15:51:22.308441 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 23 15:51:22.308441 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 23 15:51:22.308441 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 23 15:51:22.308441 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application return self.creator() Jun 23 15:51:22.308441 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 23 15:51:22.308441 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application created_value = creator( Jun 23 15:51:22.308441 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 23 15:51:22.308441 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 23 15:51:22.308441 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 23 15:51:22.308441 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: ERROR keystone.server.flask.application Jun 23 15:51:22.308441 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 634/1304] 10.222.0.46 () {56 vars in 1117 bytes} [Thu Jun 23 15:51:22 2022] GET /identity/v3/services/placement => generated 90 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 23 15:51:22.314553 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319c7d37-8e37-4fe8-bc95-40d51439d28d None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:51:22.403984 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:51:22.431808 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319c7d37-8e37-4fe8-bc95-40d51439d28d None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:51:22.472154 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67161bae-a9f9-4947-8b0a-f16fb2e28826 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:51:22.472898 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67161bae-a9f9-4947-8b0a-f16fb2e28826 None admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:51:22.473227 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67161bae-a9f9-4947-8b0a-f16fb2e28826 None admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:51:22.473566 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67161bae-a9f9-4947-8b0a-f16fb2e28826 None admin] PATH_INFO: `/v3/services` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:51:22.474345 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67161bae-a9f9-4947-8b0a-f16fb2e28826 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:51:22.474733 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67161bae-a9f9-4947-8b0a-f16fb2e28826 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:51:22.476243 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67161bae-a9f9-4947-8b0a-f16fb2e28826 None admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:51:22.482476 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 671/1305] 10.222.0.46 () {56 vars in 1116 bytes} [Thu Jun 23 15:51:22 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 15:51:22.488669 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a56981-5e00-4431-b3bb-da1bb282fe83 None admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:51:22.605649 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a56981-5e00-4431-b3bb-da1bb282fe83 None admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:51:22.646532 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681e913a-bcdf-4b23-bb5f-2e115db0ffde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:51:22.647253 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681e913a-bcdf-4b23-bb5f-2e115db0ffde None admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:51:22.647539 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681e913a-bcdf-4b23-bb5f-2e115db0ffde None admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:51:22.647816 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681e913a-bcdf-4b23-bb5f-2e115db0ffde None admin] PATH_INFO: `/v3/endpoints` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:51:22.648588 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681e913a-bcdf-4b23-bb5f-2e115db0ffde None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:51:22.650030 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681e913a-bcdf-4b23-bb5f-2e115db0ffde None admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:51:22.672981 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 635/1306] 10.222.0.46 () {60 vars in 1144 bytes} [Thu Jun 23 15:51:22 2022] POST /identity/v3/endpoints => generated 331 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 15:51:32.478953 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=30 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:51:32.668654 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:52:34.936310 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648feee3-3e7e-42cb-a64a-d32a335b9c7b None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:52:34.936803 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648feee3-3e7e-42cb-a64a-d32a335b9c7b None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:52:34.936904 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648feee3-3e7e-42cb-a64a-d32a335b9c7b None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:52:34.942741 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 672/1307] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jun 23 15:52:34 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 15:52:34.945159 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8412d70-461e-4948-849e-607969d1f0fc None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:52:34.946226 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8412d70-461e-4948-849e-607969d1f0fc None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:52:34.946938 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8412d70-461e-4948-849e-607969d1f0fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:52:35.101192 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-f8412d70-461e-4948-849e-607969d1f0fc None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:52:35.104063 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-f8412d70-461e-4948-849e-607969d1f0fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:52:35.319621 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 636/1308] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jun 23 15:52:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 376 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 15:52:35.370850 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dcd27e-ae01-40bc-9a35-f921cbace42a None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:52:35.371062 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dcd27e-ae01-40bc-9a35-f921cbace42a None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:52:35.371419 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dcd27e-ae01-40bc-9a35-f921cbace42a None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:52:35.372757 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 673/1309] 10.222.0.46 () {54 vars in 964 bytes} [Thu Jun 23 15:52:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 15:52:35.379172 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98a4522-1867-4d5a-9686-bb5809e02c7d None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:52:35.379466 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98a4522-1867-4d5a-9686-bb5809e02c7d None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:52:35.379750 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98a4522-1867-4d5a-9686-bb5809e02c7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:52:35.478119 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-d98a4522-1867-4d5a-9686-bb5809e02c7d None None] MFA Rules not processed for user `51784064951a4619ae09b9167fb0fd0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:52:35.479467 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-d98a4522-1867-4d5a-9686-bb5809e02c7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:52:35.507518 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:52:35.595895 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 637/1310] 10.222.0.46 () {58 vars in 1061 bytes} [Thu Jun 23 15:52:35 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 15:52:35.608404 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dcd27e-ae01-40bc-9a35-f921cbace42a None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:52:35.610440 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-77dcd27e-ae01-40bc-9a35-f921cbace42a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:52:35.762981 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dcd27e-ae01-40bc-9a35-f921cbace42a None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:52:35.866629 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f8b9bc-4d9d-4297-9d92-1d036a6e6179 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51784064951a4619ae09b9167fb0fd0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:52:35.868453 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f8b9bc-4d9d-4297-9d92-1d036a6e6179 service neutron] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:52:35.869059 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f8b9bc-4d9d-4297-9d92-1d036a6e6179 service neutron] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:52:35.869655 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f8b9bc-4d9d-4297-9d92-1d036a6e6179 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:52:35.873535 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-87f8b9bc-4d9d-4297-9d92-1d036a6e6179 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:52:35.949652 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:52:36.023785 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f8b9bc-4d9d-4297-9d92-1d036a6e6179 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:52:36.025788 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f8b9bc-4d9d-4297-9d92-1d036a6e6179 service neutron] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:52:36.053062 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 674/1311] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jun 23 15:52:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 445 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 15:52:38.506101 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7c9591-1411-4050-b594-56ed19e05fd3 None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:52:38.506630 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7c9591-1411-4050-b594-56ed19e05fd3 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:52:38.506630 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7c9591-1411-4050-b594-56ed19e05fd3 None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:52:38.507995 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 638/1312] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jun 23 15:52:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 15:52:38.513303 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb6314d-0bf8-4e58-96f2-28c440b07735 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:52:38.513710 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb6314d-0bf8-4e58-96f2-28c440b07735 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:52:38.513868 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb6314d-0bf8-4e58-96f2-28c440b07735 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:52:38.606805 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-0cb6314d-0bf8-4e58-96f2-28c440b07735 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:52:38.608062 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-0cb6314d-0bf8-4e58-96f2-28c440b07735 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:52:38.758644 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 675/1313] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jun 23 15:52:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 15:52:38.800059 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7c9591-1411-4050-b594-56ed19e05fd3 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:52:38.801648 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-7d7c9591-1411-4050-b594-56ed19e05fd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:52:38.919751 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7c9591-1411-4050-b594-56ed19e05fd3 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:52:39.011175 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:52:39.015353 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4412e6eb-203a-4429-bdb3-0bce4aa9fcb0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51784064951a4619ae09b9167fb0fd0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:52:39.016443 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4412e6eb-203a-4429-bdb3-0bce4aa9fcb0 service neutron] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:52:39.017013 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4412e6eb-203a-4429-bdb3-0bce4aa9fcb0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:52:39.017350 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4412e6eb-203a-4429-bdb3-0bce4aa9fcb0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:52:39.019260 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-4412e6eb-203a-4429-bdb3-0bce4aa9fcb0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:52:39.147166 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4412e6eb-203a-4429-bdb3-0bce4aa9fcb0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:52:39.148713 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4412e6eb-203a-4429-bdb3-0bce4aa9fcb0 service neutron] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:52:39.168983 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 639/1314] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jun 23 15:52:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 15:52:41.643565 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997444f3-3bdf-469b-9d22-400d479b4070 None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:52:41.644136 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997444f3-3bdf-469b-9d22-400d479b4070 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:52:41.644560 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997444f3-3bdf-469b-9d22-400d479b4070 None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:52:41.647502 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 676/1315] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jun 23 15:52:41 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 15:52:41.655446 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0e9396-ae46-458c-8331-b66cdb4becbb None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:52:41.656088 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0e9396-ae46-458c-8331-b66cdb4becbb None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:52:41.656088 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0e9396-ae46-458c-8331-b66cdb4becbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:52:41.730333 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-bf0e9396-ae46-458c-8331-b66cdb4becbb None None] MFA Rules not processed for user `160079811a0e42009ddb10073ea2cd06`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:52:41.731594 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-bf0e9396-ae46-458c-8331-b66cdb4becbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:52:41.793861 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:52:41.850600 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 640/1316] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jun 23 15:52:41 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 15:52:41.892975 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997444f3-3bdf-469b-9d22-400d479b4070 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:52:41.895239 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-997444f3-3bdf-469b-9d22-400d479b4070 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:52:41.906011 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:52:42.017053 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997444f3-3bdf-469b-9d22-400d479b4070 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:52:42.102528 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabccaf2-ac4a-45d5-af77-16649f2930d6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51784064951a4619ae09b9167fb0fd0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:52:42.103141 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabccaf2-ac4a-45d5-af77-16649f2930d6 service neutron] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:52:42.103416 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabccaf2-ac4a-45d5-af77-16649f2930d6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:52:42.103690 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabccaf2-ac4a-45d5-af77-16649f2930d6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:52:42.105453 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-cabccaf2-ac4a-45d5-af77-16649f2930d6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:52:42.177491 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:52:42.232968 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabccaf2-ac4a-45d5-af77-16649f2930d6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:52:42.234423 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabccaf2-ac4a-45d5-af77-16649f2930d6 service neutron] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:52:42.253597 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 677/1317] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jun 23 15:52:41 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 15:52:45.004697 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831558b6-3c21-4bff-821a-71780d7236fc None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:52:45.005239 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831558b6-3c21-4bff-821a-71780d7236fc None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:52:45.005239 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831558b6-3c21-4bff-821a-71780d7236fc None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:52:45.006625 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 641/1318] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jun 23 15:52:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 15:52:45.019417 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aabd972-44c7-42c0-b86f-8cc6f7f1882d None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:52:45.020223 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aabd972-44c7-42c0-b86f-8cc6f7f1882d None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:52:45.020223 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aabd972-44c7-42c0-b86f-8cc6f7f1882d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:52:45.090498 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-9aabd972-44c7-42c0-b86f-8cc6f7f1882d None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:52:45.091752 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-9aabd972-44c7-42c0-b86f-8cc6f7f1882d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:52:45.212096 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 678/1319] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jun 23 15:52:45 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 15:52:45.252487 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831558b6-3c21-4bff-821a-71780d7236fc None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:52:45.254029 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-831558b6-3c21-4bff-821a-71780d7236fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:52:45.369382 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831558b6-3c21-4bff-821a-71780d7236fc None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:52:45.497910 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c223103-33de-4902-9aa0-8eac9b8d232e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51784064951a4619ae09b9167fb0fd0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:52:45.499108 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c223103-33de-4902-9aa0-8eac9b8d232e service neutron] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:52:45.499629 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c223103-33de-4902-9aa0-8eac9b8d232e service neutron] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:52:45.500144 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c223103-33de-4902-9aa0-8eac9b8d232e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:52:45.503413 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-0c223103-33de-4902-9aa0-8eac9b8d232e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:52:45.515892 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:52:45.634838 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c223103-33de-4902-9aa0-8eac9b8d232e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:52:45.636232 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c223103-33de-4902-9aa0-8eac9b8d232e service neutron] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:52:45.657616 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 642/1320] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jun 23 15:52:45 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 15:52:48.669843 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9094caec-dc7b-4417-bb0f-0b9e92290273 None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:52:48.670394 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9094caec-dc7b-4417-bb0f-0b9e92290273 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:52:48.670394 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9094caec-dc7b-4417-bb0f-0b9e92290273 None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:52:48.672743 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 679/1321] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jun 23 15:52:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 15:52:48.677394 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ed0c31-8082-465e-8bf1-d0df807cbf87 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:52:48.678145 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ed0c31-8082-465e-8bf1-d0df807cbf87 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:52:48.678145 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ed0c31-8082-465e-8bf1-d0df807cbf87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:52:48.758411 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-d1ed0c31-8082-465e-8bf1-d0df807cbf87 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:52:48.759677 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-d1ed0c31-8082-465e-8bf1-d0df807cbf87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:52:48.822940 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:52:48.889263 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 643/1322] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jun 23 15:52:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 15:52:48.943067 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6536a673-9146-4deb-a7fa-e7e2f53fb403 None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:52:48.943356 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6536a673-9146-4deb-a7fa-e7e2f53fb403 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:52:48.943635 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6536a673-9146-4deb-a7fa-e7e2f53fb403 None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:52:48.945738 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 680/1323] 10.222.0.46 () {54 vars in 964 bytes} [Thu Jun 23 15:52:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 15:52:48.955974 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13da257c-7657-41eb-9683-65426314b5a8 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:52:48.956261 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13da257c-7657-41eb-9683-65426314b5a8 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:52:48.956643 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13da257c-7657-41eb-9683-65426314b5a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:52:49.057435 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-13da257c-7657-41eb-9683-65426314b5a8 None None] MFA Rules not processed for user `51784064951a4619ae09b9167fb0fd0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:52:49.058736 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-13da257c-7657-41eb-9683-65426314b5a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:52:49.144486 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 644/1324] 10.222.0.46 () {58 vars in 1061 bytes} [Thu Jun 23 15:52:48 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 15:52:49.154294 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6536a673-9146-4deb-a7fa-e7e2f53fb403 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:52:49.156125 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-6536a673-9146-4deb-a7fa-e7e2f53fb403 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:52:49.178307 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:52:49.311074 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6536a673-9146-4deb-a7fa-e7e2f53fb403 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:52:49.422201 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd5e863-4c3a-4e15-82f5-1887cd081aea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51784064951a4619ae09b9167fb0fd0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:52:49.423302 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd5e863-4c3a-4e15-82f5-1887cd081aea service neutron] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:52:49.423803 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd5e863-4c3a-4e15-82f5-1887cd081aea service neutron] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:52:49.424298 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd5e863-4c3a-4e15-82f5-1887cd081aea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:52:49.427125 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-bfd5e863-4c3a-4e15-82f5-1887cd081aea service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:52:49.542370 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:52:49.596167 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd5e863-4c3a-4e15-82f5-1887cd081aea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:52:49.597930 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd5e863-4c3a-4e15-82f5-1887cd081aea service neutron] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:52:49.620338 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 681/1325] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jun 23 15:52:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 467 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 15:52:52.087972 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f205dd1-d2ec-46cb-92b5-c363b7fbad7a None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:52:52.088594 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f205dd1-d2ec-46cb-92b5-c363b7fbad7a None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:52:52.088594 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f205dd1-d2ec-46cb-92b5-c363b7fbad7a None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:52:52.090226 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 645/1326] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jun 23 15:52:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 15:52:52.096216 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ca25c5-816f-47d0-b183-42abbb79a2b4 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:52:52.096747 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ca25c5-816f-47d0-b183-42abbb79a2b4 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:52:52.096869 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ca25c5-816f-47d0-b183-42abbb79a2b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:52:52.174012 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-84ca25c5-816f-47d0-b183-42abbb79a2b4 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:52:52.175296 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-84ca25c5-816f-47d0-b183-42abbb79a2b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:52:52.298555 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 682/1327] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jun 23 15:52:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 15:52:52.336356 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f205dd1-d2ec-46cb-92b5-c363b7fbad7a None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:52:52.337964 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-2f205dd1-d2ec-46cb-92b5-c363b7fbad7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:52:52.412424 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:52:52.465158 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f205dd1-d2ec-46cb-92b5-c363b7fbad7a None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:52:52.553809 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6afbfc-81db-4705-b037-fa87a64e9ff8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51784064951a4619ae09b9167fb0fd0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:52:52.555129 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6afbfc-81db-4705-b037-fa87a64e9ff8 service neutron] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:52:52.555356 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6afbfc-81db-4705-b037-fa87a64e9ff8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:52:52.555594 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6afbfc-81db-4705-b037-fa87a64e9ff8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:52:52.557407 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-ba6afbfc-81db-4705-b037-fa87a64e9ff8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:52:52.702190 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:52:52.705067 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6afbfc-81db-4705-b037-fa87a64e9ff8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:52:52.706898 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6afbfc-81db-4705-b037-fa87a64e9ff8 service neutron] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:52:52.728975 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 646/1328] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jun 23 15:52:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 15:52:54.925493 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1755ce1f-90d9-4807-9f80-455ac62cdc2b None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:52:54.926069 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1755ce1f-90d9-4807-9f80-455ac62cdc2b None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:52:54.926235 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1755ce1f-90d9-4807-9f80-455ac62cdc2b None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:52:54.927625 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 683/1329] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jun 23 15:52:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 15:52:54.933639 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad11d40d-9267-4c1f-a0b2-37fe8a4b867e None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:52:54.934215 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad11d40d-9267-4c1f-a0b2-37fe8a4b867e None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:52:54.934215 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad11d40d-9267-4c1f-a0b2-37fe8a4b867e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:52:55.014146 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-ad11d40d-9267-4c1f-a0b2-37fe8a4b867e None None] MFA Rules not processed for user `160079811a0e42009ddb10073ea2cd06`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:52:55.015418 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-ad11d40d-9267-4c1f-a0b2-37fe8a4b867e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:52:55.151145 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 647/1330] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jun 23 15:52:54 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 15:52:55.197143 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1755ce1f-90d9-4807-9f80-455ac62cdc2b None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:52:55.200006 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-1755ce1f-90d9-4807-9f80-455ac62cdc2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:52:55.235489 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:52:55.330739 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1755ce1f-90d9-4807-9f80-455ac62cdc2b None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:52:55.412925 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824bca44-5019-4fad-a87f-038c854294ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51784064951a4619ae09b9167fb0fd0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:52:55.413502 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824bca44-5019-4fad-a87f-038c854294ef service neutron] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:52:55.413789 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824bca44-5019-4fad-a87f-038c854294ef service neutron] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:52:55.414066 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824bca44-5019-4fad-a87f-038c854294ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:52:55.415972 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-824bca44-5019-4fad-a87f-038c854294ef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:52:55.504316 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:52:55.542526 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824bca44-5019-4fad-a87f-038c854294ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:52:55.544029 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824bca44-5019-4fad-a87f-038c854294ef service neutron] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:52:55.564455 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 684/1331] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jun 23 15:52:55 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 15:52:58.066757 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2929aea-9065-4bbb-8dc7-56714374a7a2 None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:52:58.067345 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2929aea-9065-4bbb-8dc7-56714374a7a2 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:52:58.067345 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2929aea-9065-4bbb-8dc7-56714374a7a2 None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:52:58.068731 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 648/1332] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jun 23 15:52:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 15:52:58.075411 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84dd91c0-f7be-40ce-a3d1-605a40063dfc None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:52:58.075924 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84dd91c0-f7be-40ce-a3d1-605a40063dfc None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:52:58.076039 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84dd91c0-f7be-40ce-a3d1-605a40063dfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:52:58.155854 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-84dd91c0-f7be-40ce-a3d1-605a40063dfc None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:52:58.157104 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-84dd91c0-f7be-40ce-a3d1-605a40063dfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:52:58.269882 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 685/1333] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jun 23 15:52:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 15:52:58.308024 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2929aea-9065-4bbb-8dc7-56714374a7a2 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:52:58.309782 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-d2929aea-9065-4bbb-8dc7-56714374a7a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:52:58.398871 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:52:58.440769 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2929aea-9065-4bbb-8dc7-56714374a7a2 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:52:58.532640 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfe8424-2ade-497f-b79f-9222416fb93f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51784064951a4619ae09b9167fb0fd0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:52:58.533771 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfe8424-2ade-497f-b79f-9222416fb93f service neutron] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:52:58.534283 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfe8424-2ade-497f-b79f-9222416fb93f service neutron] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:52:58.534805 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfe8424-2ade-497f-b79f-9222416fb93f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:52:58.536993 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-7cfe8424-2ade-497f-b79f-9222416fb93f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:52:58.678719 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfe8424-2ade-497f-b79f-9222416fb93f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:52:58.680196 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfe8424-2ade-497f-b79f-9222416fb93f service neutron] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:52:58.692253 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:52:58.702087 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 649/1334] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jun 23 15:52:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 15:53:00.860847 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a295aa9-ea2a-4718-8596-705dd8349407 None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:53:00.861567 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a295aa9-ea2a-4718-8596-705dd8349407 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:53:00.861567 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a295aa9-ea2a-4718-8596-705dd8349407 None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:53:00.862841 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 686/1335] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jun 23 15:53:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 15:53:00.869227 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cf960d-782b-48e8-8f0b-c0e0f4bd6ca2 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:53:00.869785 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cf960d-782b-48e8-8f0b-c0e0f4bd6ca2 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:53:00.869785 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cf960d-782b-48e8-8f0b-c0e0f4bd6ca2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:53:00.944280 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-73cf960d-782b-48e8-8f0b-c0e0f4bd6ca2 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:53:00.946078 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-73cf960d-782b-48e8-8f0b-c0e0f4bd6ca2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:53:01.062108 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 650/1336] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jun 23 15:53:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 15:53:01.109812 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a295aa9-ea2a-4718-8596-705dd8349407 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:53:01.112297 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-1a295aa9-ea2a-4718-8596-705dd8349407 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:53:01.158942 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:53:01.234317 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a295aa9-ea2a-4718-8596-705dd8349407 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:53:01.319263 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66873249-c4a9-4069-a51f-b9289f61b6e1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51784064951a4619ae09b9167fb0fd0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:53:01.320082 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66873249-c4a9-4069-a51f-b9289f61b6e1 service neutron] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:53:01.320373 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66873249-c4a9-4069-a51f-b9289f61b6e1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:53:01.320648 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66873249-c4a9-4069-a51f-b9289f61b6e1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:53:01.322582 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-66873249-c4a9-4069-a51f-b9289f61b6e1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:53:01.422745 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:53:01.452104 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66873249-c4a9-4069-a51f-b9289f61b6e1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:53:01.453588 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66873249-c4a9-4069-a51f-b9289f61b6e1 service neutron] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:53:01.473583 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 687/1337] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jun 23 15:53:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 15:53:04.000557 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925c0c5a-0bfd-4157-95f6-9dc239aca565 None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:53:04.001146 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925c0c5a-0bfd-4157-95f6-9dc239aca565 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:53:04.001308 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925c0c5a-0bfd-4157-95f6-9dc239aca565 None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:53:04.002816 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 651/1338] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jun 23 15:53:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 15:53:04.009150 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13e4cd7-b49a-46fb-b5eb-ffd9ddf6ab3f None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:53:04.009720 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13e4cd7-b49a-46fb-b5eb-ffd9ddf6ab3f None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:53:04.009720 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13e4cd7-b49a-46fb-b5eb-ffd9ddf6ab3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:53:04.085817 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-c13e4cd7-b49a-46fb-b5eb-ffd9ddf6ab3f None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:53:04.087093 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-c13e4cd7-b49a-46fb-b5eb-ffd9ddf6ab3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:53:04.202977 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 688/1339] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jun 23 15:53:04 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 15:53:04.241297 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925c0c5a-0bfd-4157-95f6-9dc239aca565 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:53:04.244169 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-925c0c5a-0bfd-4157-95f6-9dc239aca565 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:53:04.338139 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:53:04.370363 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925c0c5a-0bfd-4157-95f6-9dc239aca565 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:53:04.454898 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d359adc5-56ea-49f3-9f46-59afd07de9df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51784064951a4619ae09b9167fb0fd0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:53:04.455703 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d359adc5-56ea-49f3-9f46-59afd07de9df service neutron] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:53:04.456059 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d359adc5-56ea-49f3-9f46-59afd07de9df service neutron] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:53:04.456381 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d359adc5-56ea-49f3-9f46-59afd07de9df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:53:04.458205 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-d359adc5-56ea-49f3-9f46-59afd07de9df service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:53:04.583694 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d359adc5-56ea-49f3-9f46-59afd07de9df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:53:04.585874 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d359adc5-56ea-49f3-9f46-59afd07de9df service neutron] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:53:04.606403 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 652/1340] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jun 23 15:53:04 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 15:53:09.475914 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2990a921-48ae-4133-b1ef-6f57cfb1a278 None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:53:09.476555 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2990a921-48ae-4133-b1ef-6f57cfb1a278 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:53:09.476882 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2990a921-48ae-4133-b1ef-6f57cfb1a278 None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:53:09.479736 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 689/1341] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jun 23 15:53:09 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 15:53:09.486137 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bb6a3e-8921-4e7b-8563-0a380f9cf015 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:53:09.486137 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bb6a3e-8921-4e7b-8563-0a380f9cf015 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:53:09.486137 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bb6a3e-8921-4e7b-8563-0a380f9cf015 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:53:09.494514 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:53:09.595084 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-c8bb6a3e-8921-4e7b-8563-0a380f9cf015 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:53:09.596664 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-c8bb6a3e-8921-4e7b-8563-0a380f9cf015 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:53:09.729299 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 653/1342] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jun 23 15:53:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 15:53:09.770455 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2990a921-48ae-4133-b1ef-6f57cfb1a278 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:53:09.773586 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-2990a921-48ae-4133-b1ef-6f57cfb1a278 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:53:09.832648 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:53:09.888812 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2990a921-48ae-4133-b1ef-6f57cfb1a278 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:53:09.892637 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80083a30-d515-4422-9f36-49c33759a221 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51784064951a4619ae09b9167fb0fd0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:53:09.893315 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80083a30-d515-4422-9f36-49c33759a221 service neutron] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:53:09.893597 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80083a30-d515-4422-9f36-49c33759a221 service neutron] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:53:09.893896 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80083a30-d515-4422-9f36-49c33759a221 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:53:09.915643 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80083a30-d515-4422-9f36-49c33759a221 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:53:09.917123 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80083a30-d515-4422-9f36-49c33759a221 service neutron] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:53:09.929300 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 690/1343] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jun 23 15:53:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 15:53:13.380001 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112672e0-3e40-4973-ab01-18bdaac66002 None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:53:13.380904 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112672e0-3e40-4973-ab01-18bdaac66002 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:53:13.380904 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112672e0-3e40-4973-ab01-18bdaac66002 None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:53:13.384278 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 654/1344] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jun 23 15:53:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 15:53:13.392382 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0c07f2-e7ad-46bf-8a8a-4603f0ecda09 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:53:13.393160 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0c07f2-e7ad-46bf-8a8a-4603f0ecda09 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:53:13.393431 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0c07f2-e7ad-46bf-8a8a-4603f0ecda09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:53:13.446662 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-7e0c07f2-e7ad-46bf-8a8a-4603f0ecda09 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:53:13.449050 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-7e0c07f2-e7ad-46bf-8a8a-4603f0ecda09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:53:13.587442 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 691/1345] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jun 23 15:53:13 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 15:53:13.632195 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112672e0-3e40-4973-ab01-18bdaac66002 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:53:13.646452 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112672e0-3e40-4973-ab01-18bdaac66002 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:53:13.651332 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b83e61-f647-44f1-83d4-56edbe065b1a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51784064951a4619ae09b9167fb0fd0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:53:13.652226 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b83e61-f647-44f1-83d4-56edbe065b1a service neutron] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:53:13.652721 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b83e61-f647-44f1-83d4-56edbe065b1a service neutron] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:53:13.653209 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b83e61-f647-44f1-83d4-56edbe065b1a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:53:13.672201 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b83e61-f647-44f1-83d4-56edbe065b1a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:53:13.674025 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b83e61-f647-44f1-83d4-56edbe065b1a service neutron] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:53:13.687201 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 655/1346] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jun 23 15:53:13 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 15:53:19.027625 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b407e591-1427-409c-9bcd-aac38e56ecec None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:53:19.028337 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b407e591-1427-409c-9bcd-aac38e56ecec None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:53:19.028453 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b407e591-1427-409c-9bcd-aac38e56ecec None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:53:19.031081 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 692/1347] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jun 23 15:53:19 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 15:53:19.039993 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8c8fe2-93d3-4220-86f4-4cb32b745700 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:53:19.040662 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8c8fe2-93d3-4220-86f4-4cb32b745700 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:53:19.040926 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8c8fe2-93d3-4220-86f4-4cb32b745700 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:53:19.070588 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-7a8c8fe2-93d3-4220-86f4-4cb32b745700 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:53:19.071901 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-7a8c8fe2-93d3-4220-86f4-4cb32b745700 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:53:19.082394 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 656/1348] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jun 23 15:53:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 15:53:19.130243 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b407e591-1427-409c-9bcd-aac38e56ecec None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:53:19.146822 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b407e591-1427-409c-9bcd-aac38e56ecec None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:53:19.151820 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ba668f-5f64-42a1-8798-44ee00638d8a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51784064951a4619ae09b9167fb0fd0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:53:19.153398 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ba668f-5f64-42a1-8798-44ee00638d8a service neutron] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:53:19.153398 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ba668f-5f64-42a1-8798-44ee00638d8a service neutron] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:53:19.155201 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ba668f-5f64-42a1-8798-44ee00638d8a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:53:19.171697 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ba668f-5f64-42a1-8798-44ee00638d8a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:53:19.173803 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ba668f-5f64-42a1-8798-44ee00638d8a service neutron] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:53:19.188844 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 693/1349] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jun 23 15:53:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 15:53:25.994377 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e173f3-ebf4-4c54-8ce5-5d1a894bf3e5 None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:53:25.995410 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e173f3-ebf4-4c54-8ce5-5d1a894bf3e5 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:53:25.995919 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e173f3-ebf4-4c54-8ce5-5d1a894bf3e5 None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:53:25.998021 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 657/1350] 10.222.0.46 () {54 vars in 921 bytes} [Thu Jun 23 15:53:25 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 15:53:26.004459 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75289d2-36ef-49e3-af75-997034ceef77 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:53:26.005518 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75289d2-36ef-49e3-af75-997034ceef77 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:53:26.006070 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75289d2-36ef-49e3-af75-997034ceef77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:53:26.116956 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-b75289d2-36ef-49e3-af75-997034ceef77 None None] MFA Rules not processed for user `3f66988b73644fe38b9849663ee70f25`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:53:26.118273 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-b75289d2-36ef-49e3-af75-997034ceef77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:53:26.152729 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:53:26.251895 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 694/1351] 10.222.0.46 () {58 vars in 1018 bytes} [Thu Jun 23 15:53:26 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 15:53:29.062401 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=72 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:53:31.298176 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a64a673-7190-45be-a510-07043f40282f None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:53:31.299154 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a64a673-7190-45be-a510-07043f40282f None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:53:31.299154 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a64a673-7190-45be-a510-07043f40282f None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:53:31.300582 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 658/1352] 10.222.0.46 () {54 vars in 921 bytes} [Thu Jun 23 15:53:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 15:53:31.307916 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9fcfd7-f9be-4b05-9415-4e2653e40544 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:53:31.309460 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9fcfd7-f9be-4b05-9415-4e2653e40544 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:53:31.309945 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9fcfd7-f9be-4b05-9415-4e2653e40544 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:53:31.349726 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-1c9fcfd7-f9be-4b05-9415-4e2653e40544 None None] MFA Rules not processed for user `3f66988b73644fe38b9849663ee70f25`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:53:31.352423 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-1c9fcfd7-f9be-4b05-9415-4e2653e40544 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:53:31.372633 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 695/1353] 10.222.0.46 () {58 vars in 1018 bytes} [Thu Jun 23 15:53:31 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 15:53:33.183590 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e8d8e3-16ff-46f0-a075-26c7c984cbd7 None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:53:33.184134 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e8d8e3-16ff-46f0-a075-26c7c984cbd7 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:53:33.184244 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e8d8e3-16ff-46f0-a075-26c7c984cbd7 None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:53:33.185662 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 659/1354] 10.222.0.46 () {54 vars in 921 bytes} [Thu Jun 23 15:53:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 15:53:33.192008 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2054fdb2-5b0e-43cd-a9e3-25874e229da8 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:53:33.192545 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2054fdb2-5b0e-43cd-a9e3-25874e229da8 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:53:33.192545 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2054fdb2-5b0e-43cd-a9e3-25874e229da8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:53:33.213346 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-2054fdb2-5b0e-43cd-a9e3-25874e229da8 None None] MFA Rules not processed for user `3f66988b73644fe38b9849663ee70f25`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:53:33.214637 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-2054fdb2-5b0e-43cd-a9e3-25874e229da8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:53:33.227831 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 696/1355] 10.222.0.46 () {58 vars in 1018 bytes} [Thu Jun 23 15:53:33 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 15:53:37.096014 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a266d5a-74ce-4d2e-87ca-46f75eaada2a None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:53:37.097029 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a266d5a-74ce-4d2e-87ca-46f75eaada2a None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:53:37.097523 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a266d5a-74ce-4d2e-87ca-46f75eaada2a None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:53:37.099400 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 660/1356] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jun 23 15:53:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 15:53:37.105934 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1b2763-8910-473f-8131-4ae039ded3ce None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:53:37.106773 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1b2763-8910-473f-8131-4ae039ded3ce None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:53:37.107053 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1b2763-8910-473f-8131-4ae039ded3ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:53:37.139209 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-bc1b2763-8910-473f-8131-4ae039ded3ce None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:53:37.140376 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-bc1b2763-8910-473f-8131-4ae039ded3ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:53:37.150239 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 697/1357] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jun 23 15:53:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 15:53:37.214972 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c45259-a9d8-4162-a4ee-a37fd47e2ca9 None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:53:37.214972 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c45259-a9d8-4162-a4ee-a37fd47e2ca9 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:53:37.215360 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c45259-a9d8-4162-a4ee-a37fd47e2ca9 None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:53:37.217088 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 661/1358] 10.222.0.46 () {54 vars in 961 bytes} [Thu Jun 23 15:53:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 15:53:37.228006 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2608d2-ecc2-4c70-8173-58a52688e4e3 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:53:37.228225 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2608d2-ecc2-4c70-8173-58a52688e4e3 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:53:37.228710 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2608d2-ecc2-4c70-8173-58a52688e4e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:53:37.293623 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-fe2608d2-ecc2-4c70-8173-58a52688e4e3 None None] MFA Rules not processed for user `705115d19bab424d894bea8b1b48a7f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:53:37.295995 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-fe2608d2-ecc2-4c70-8173-58a52688e4e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:53:37.404367 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 698/1359] 10.222.0.46 () {58 vars in 1058 bytes} [Thu Jun 23 15:53:37 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 15:53:37.412060 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c45259-a9d8-4162-a4ee-a37fd47e2ca9 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:53:37.429270 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c45259-a9d8-4162-a4ee-a37fd47e2ca9 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:53:37.435244 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf58691-6452-4b74-8980-f11f67ed317c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '705115d19bab424d894bea8b1b48a7f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:53:37.436421 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf58691-6452-4b74-8980-f11f67ed317c service nova] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:53:37.436852 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf58691-6452-4b74-8980-f11f67ed317c service nova] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:53:37.437284 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf58691-6452-4b74-8980-f11f67ed317c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:53:37.446671 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf58691-6452-4b74-8980-f11f67ed317c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:53:37.448586 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf58691-6452-4b74-8980-f11f67ed317c service nova] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:53:37.459671 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 662/1360] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jun 23 15:53:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 15:53:39.863574 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9115eac9-736b-4e5f-9c82-d415fc0c68a7 None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:53:39.864221 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9115eac9-736b-4e5f-9c82-d415fc0c68a7 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:53:39.864499 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9115eac9-736b-4e5f-9c82-d415fc0c68a7 None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:53:39.865784 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 699/1361] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jun 23 15:53:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 15:53:39.872049 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8661b65c-76fc-43dc-9b16-4660f58f897f None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:53:39.872633 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8661b65c-76fc-43dc-9b16-4660f58f897f None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:53:39.872633 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8661b65c-76fc-43dc-9b16-4660f58f897f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:53:39.891536 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-8661b65c-76fc-43dc-9b16-4660f58f897f None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:53:39.892786 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-8661b65c-76fc-43dc-9b16-4660f58f897f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:53:39.906041 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 663/1362] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jun 23 15:53:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 15:53:39.951190 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9115eac9-736b-4e5f-9c82-d415fc0c68a7 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:53:39.973426 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9115eac9-736b-4e5f-9c82-d415fc0c68a7 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:53:39.981242 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f46c76-37be-4edf-a83f-f725783e8610 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '705115d19bab424d894bea8b1b48a7f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:53:39.982358 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f46c76-37be-4edf-a83f-f725783e8610 service nova] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:53:39.982847 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f46c76-37be-4edf-a83f-f725783e8610 service nova] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:53:39.983305 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f46c76-37be-4edf-a83f-f725783e8610 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:53:39.994733 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f46c76-37be-4edf-a83f-f725783e8610 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:53:39.996626 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f46c76-37be-4edf-a83f-f725783e8610 service nova] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:53:40.013743 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 700/1363] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jun 23 15:53:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 15:53:42.613937 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0ead92-0537-417f-99e4-f83107136458 None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:53:42.614447 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0ead92-0537-417f-99e4-f83107136458 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:53:42.614531 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0ead92-0537-417f-99e4-f83107136458 None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:53:42.618227 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 664/1364] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jun 23 15:53:42 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 15:53:42.624835 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c2c57d-9632-4b58-acb0-af990f54cabb None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:53:42.625742 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c2c57d-9632-4b58-acb0-af990f54cabb None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:53:42.626091 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c2c57d-9632-4b58-acb0-af990f54cabb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:53:42.647450 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-b0c2c57d-9632-4b58-acb0-af990f54cabb None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:53:42.648654 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-b0c2c57d-9632-4b58-acb0-af990f54cabb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:53:42.656609 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 701/1365] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jun 23 15:53:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 15:53:42.702543 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5d5e64-080c-4a9e-989b-7e05b2ec0f52 None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:53:42.702882 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5d5e64-080c-4a9e-989b-7e05b2ec0f52 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:53:42.703161 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5d5e64-080c-4a9e-989b-7e05b2ec0f52 None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:53:42.704520 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 665/1366] 10.222.0.46 () {54 vars in 961 bytes} [Thu Jun 23 15:53:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 15:53:42.710713 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023b5a85-f44e-4362-85bc-55b3998d4dc9 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:53:42.711023 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023b5a85-f44e-4362-85bc-55b3998d4dc9 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:53:42.711269 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023b5a85-f44e-4362-85bc-55b3998d4dc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:53:42.730718 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-023b5a85-f44e-4362-85bc-55b3998d4dc9 None None] MFA Rules not processed for user `705115d19bab424d894bea8b1b48a7f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:53:42.731908 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-023b5a85-f44e-4362-85bc-55b3998d4dc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:53:42.741133 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 702/1367] 10.222.0.46 () {58 vars in 1058 bytes} [Thu Jun 23 15:53:42 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 15:53:42.749382 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5d5e64-080c-4a9e-989b-7e05b2ec0f52 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:53:42.763744 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5d5e64-080c-4a9e-989b-7e05b2ec0f52 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:53:42.769236 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409d91ad-c737-41e0-86ee-cc7779032ce6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '705115d19bab424d894bea8b1b48a7f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:53:42.769881 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409d91ad-c737-41e0-86ee-cc7779032ce6 service nova] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:53:42.770434 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409d91ad-c737-41e0-86ee-cc7779032ce6 service nova] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:53:42.770735 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409d91ad-c737-41e0-86ee-cc7779032ce6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:53:42.779721 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409d91ad-c737-41e0-86ee-cc7779032ce6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:53:42.781381 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409d91ad-c737-41e0-86ee-cc7779032ce6 service nova] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:53:42.791975 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 666/1368] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jun 23 15:53:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 15:53:44.961039 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f5a8b4-b99b-4a62-a434-1cc914ab10d6 None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:53:44.961655 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f5a8b4-b99b-4a62-a434-1cc914ab10d6 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:53:44.961655 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f5a8b4-b99b-4a62-a434-1cc914ab10d6 None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:53:44.963110 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 703/1369] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jun 23 15:53:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 15:53:44.969359 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3ccece-4eec-49b9-b724-daf40d820c60 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:53:44.969903 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3ccece-4eec-49b9-b724-daf40d820c60 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:53:44.969903 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3ccece-4eec-49b9-b724-daf40d820c60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:53:44.993843 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-2e3ccece-4eec-49b9-b724-daf40d820c60 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:53:44.995201 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-2e3ccece-4eec-49b9-b724-daf40d820c60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:53:45.003658 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 667/1370] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jun 23 15:53:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 15:53:45.049054 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f5a8b4-b99b-4a62-a434-1cc914ab10d6 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:53:45.067259 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f5a8b4-b99b-4a62-a434-1cc914ab10d6 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:53:45.074152 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab67c1d-749d-406a-9241-aa7fb8c905af service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '705115d19bab424d894bea8b1b48a7f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:53:45.074855 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab67c1d-749d-406a-9241-aa7fb8c905af service nova] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:53:45.075238 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab67c1d-749d-406a-9241-aa7fb8c905af service nova] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:53:45.075521 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab67c1d-749d-406a-9241-aa7fb8c905af service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:53:45.085077 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab67c1d-749d-406a-9241-aa7fb8c905af service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:53:45.086709 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab67c1d-749d-406a-9241-aa7fb8c905af service nova] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:53:45.097455 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 704/1371] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jun 23 15:53:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 15:53:45.839891 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0087990-4631-44e3-89c4-0d95cc600c5f None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:53:45.840906 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0087990-4631-44e3-89c4-0d95cc600c5f None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:53:45.841731 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0087990-4631-44e3-89c4-0d95cc600c5f None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:53:45.844427 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 668/1372] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jun 23 15:53:45 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 15:53:45.851063 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efd4b0b-bed5-4145-a3cb-f8a883816e06 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:53:45.851401 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efd4b0b-bed5-4145-a3cb-f8a883816e06 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:53:45.851684 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efd4b0b-bed5-4145-a3cb-f8a883816e06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:53:45.870892 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-3efd4b0b-bed5-4145-a3cb-f8a883816e06 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:53:45.872108 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-3efd4b0b-bed5-4145-a3cb-f8a883816e06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:53:45.882805 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 705/1373] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jun 23 15:53:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 15:53:46.259279 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014439db-2e5f-4639-a51c-f05d2df57198 None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:53:46.259279 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014439db-2e5f-4639-a51c-f05d2df57198 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:53:46.259279 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014439db-2e5f-4639-a51c-f05d2df57198 None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:53:46.262605 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 669/1374] 10.222.0.46 () {54 vars in 963 bytes} [Thu Jun 23 15:53:46 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 15:53:46.268784 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab335c22-8d2d-4194-aaaf-699c0d7fac22 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:53:46.269263 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab335c22-8d2d-4194-aaaf-699c0d7fac22 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:53:46.269535 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab335c22-8d2d-4194-aaaf-699c0d7fac22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:53:46.277380 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:53:46.358402 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-ab335c22-8d2d-4194-aaaf-699c0d7fac22 None None] MFA Rules not processed for user `ce18eaaba3a6477684fff1fd98ec8b51`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:53:46.360537 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-ab335c22-8d2d-4194-aaaf-699c0d7fac22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:53:46.482973 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 706/1375] 10.222.0.46 () {58 vars in 1060 bytes} [Thu Jun 23 15:53:46 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 15:53:46.492129 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014439db-2e5f-4639-a51c-f05d2df57198 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:53:46.509376 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014439db-2e5f-4639-a51c-f05d2df57198 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:53:46.514380 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3288cf9a-15a6-46f2-a4e5-13c03e268074 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce18eaaba3a6477684fff1fd98ec8b51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:53:46.515439 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3288cf9a-15a6-46f2-a4e5-13c03e268074 service cinder] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:53:46.515838 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3288cf9a-15a6-46f2-a4e5-13c03e268074 service cinder] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:53:46.516271 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3288cf9a-15a6-46f2-a4e5-13c03e268074 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:53:46.530162 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3288cf9a-15a6-46f2-a4e5-13c03e268074 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:53:46.532840 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3288cf9a-15a6-46f2-a4e5-13c03e268074 service cinder] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:53:46.549864 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 670/1376] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jun 23 15:53:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 15:53:47.415819 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66d8587-bc5e-47fa-b1cd-5f2e2d2151ef None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:53:47.415819 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66d8587-bc5e-47fa-b1cd-5f2e2d2151ef None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:53:47.415819 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66d8587-bc5e-47fa-b1cd-5f2e2d2151ef None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:53:47.421733 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 707/1377] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jun 23 15:53:47 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 15:53:47.421914 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d70365-ecad-4a5f-ad80-6ea7ef898617 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:53:47.422665 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d70365-ecad-4a5f-ad80-6ea7ef898617 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:53:47.422665 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d70365-ecad-4a5f-ad80-6ea7ef898617 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:53:47.456972 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-34d70365-ecad-4a5f-ad80-6ea7ef898617 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:53:47.458939 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-34d70365-ecad-4a5f-ad80-6ea7ef898617 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:53:47.479762 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 671/1378] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jun 23 15:53:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 15:53:47.564166 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66d8587-bc5e-47fa-b1cd-5f2e2d2151ef None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:53:47.589014 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66d8587-bc5e-47fa-b1cd-5f2e2d2151ef None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:53:47.596672 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8393abac-6eee-48ad-8a05-c05828bec22f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '705115d19bab424d894bea8b1b48a7f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:53:47.597477 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8393abac-6eee-48ad-8a05-c05828bec22f service nova] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:53:47.597849 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8393abac-6eee-48ad-8a05-c05828bec22f service nova] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:53:47.598203 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8393abac-6eee-48ad-8a05-c05828bec22f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:53:47.608604 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8393abac-6eee-48ad-8a05-c05828bec22f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:53:47.610225 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8393abac-6eee-48ad-8a05-c05828bec22f service nova] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:53:47.622008 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 708/1379] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jun 23 15:53:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 15:53:49.775183 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88770aab-9414-42df-8abf-1b2cfe26e843 None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:53:49.775691 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88770aab-9414-42df-8abf-1b2cfe26e843 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:53:49.775827 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88770aab-9414-42df-8abf-1b2cfe26e843 None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:53:49.777117 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 672/1380] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jun 23 15:53:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 15:53:49.782489 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290d5aee-8a15-432d-8d23-07a7708d7a3d None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:53:49.782963 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290d5aee-8a15-432d-8d23-07a7708d7a3d None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:53:49.783228 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290d5aee-8a15-432d-8d23-07a7708d7a3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:53:49.801831 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-290d5aee-8a15-432d-8d23-07a7708d7a3d None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:53:49.803062 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-290d5aee-8a15-432d-8d23-07a7708d7a3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:53:49.817283 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 709/1381] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jun 23 15:53:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 15:53:49.876421 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88770aab-9414-42df-8abf-1b2cfe26e843 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:53:49.891947 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88770aab-9414-42df-8abf-1b2cfe26e843 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:53:49.899005 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe1ce23-711e-4338-b6c8-f26fac4b762c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '705115d19bab424d894bea8b1b48a7f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:53:49.899683 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe1ce23-711e-4338-b6c8-f26fac4b762c service nova] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:53:49.900118 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe1ce23-711e-4338-b6c8-f26fac4b762c service nova] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:53:49.900404 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe1ce23-711e-4338-b6c8-f26fac4b762c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:53:49.910452 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe1ce23-711e-4338-b6c8-f26fac4b762c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:53:49.912265 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe1ce23-711e-4338-b6c8-f26fac4b762c service nova] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:53:49.936433 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 673/1382] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jun 23 15:53:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 15:53:52.095817 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168c0abb-42cf-421c-8fd6-7db95424245e None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:53:52.096395 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168c0abb-42cf-421c-8fd6-7db95424245e None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:53:52.096395 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168c0abb-42cf-421c-8fd6-7db95424245e None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:53:52.097754 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 710/1383] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jun 23 15:53:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 15:53:52.105856 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e825642-641e-47ad-aa15-8b91a9e5fe18 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:53:52.107363 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e825642-641e-47ad-aa15-8b91a9e5fe18 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:53:52.107660 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e825642-641e-47ad-aa15-8b91a9e5fe18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:53:52.126562 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-3e825642-641e-47ad-aa15-8b91a9e5fe18 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:53:52.127824 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-3e825642-641e-47ad-aa15-8b91a9e5fe18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:53:52.135900 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 674/1384] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jun 23 15:53:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 15:53:52.178989 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168c0abb-42cf-421c-8fd6-7db95424245e None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:53:52.194129 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168c0abb-42cf-421c-8fd6-7db95424245e None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:53:52.199950 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930d6e1d-7068-498a-9d72-421b10d20da3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '705115d19bab424d894bea8b1b48a7f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:53:52.200852 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930d6e1d-7068-498a-9d72-421b10d20da3 service nova] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:53:52.201210 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930d6e1d-7068-498a-9d72-421b10d20da3 service nova] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:53:52.201567 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930d6e1d-7068-498a-9d72-421b10d20da3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:53:52.210787 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930d6e1d-7068-498a-9d72-421b10d20da3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:53:52.212473 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930d6e1d-7068-498a-9d72-421b10d20da3 service nova] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:53:52.222969 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 711/1385] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jun 23 15:53:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 15:53:54.353760 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2545fa75-94d7-4fa0-af81-257806bcd7d7 None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:53:54.355020 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2545fa75-94d7-4fa0-af81-257806bcd7d7 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:53:54.355754 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2545fa75-94d7-4fa0-af81-257806bcd7d7 None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:53:54.358876 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 675/1386] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jun 23 15:53:54 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 15:53:54.364652 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b3e121-a7d9-4c05-846f-878ffaa19ecf None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:53:54.365357 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b3e121-a7d9-4c05-846f-878ffaa19ecf None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:53:54.365357 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b3e121-a7d9-4c05-846f-878ffaa19ecf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:53:54.384758 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-b3b3e121-a7d9-4c05-846f-878ffaa19ecf None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:53:54.385980 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-b3b3e121-a7d9-4c05-846f-878ffaa19ecf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:53:54.394589 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 712/1387] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jun 23 15:53:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 15:53:54.445495 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2545fa75-94d7-4fa0-af81-257806bcd7d7 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:53:54.464266 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2545fa75-94d7-4fa0-af81-257806bcd7d7 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:53:54.470150 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2eeb52e-2438-4d6b-9969-5e4f5a9ad9e0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '705115d19bab424d894bea8b1b48a7f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:53:54.471145 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2eeb52e-2438-4d6b-9969-5e4f5a9ad9e0 service nova] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:53:54.471145 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2eeb52e-2438-4d6b-9969-5e4f5a9ad9e0 service nova] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:53:54.471145 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2eeb52e-2438-4d6b-9969-5e4f5a9ad9e0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:53:54.481128 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2eeb52e-2438-4d6b-9969-5e4f5a9ad9e0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:53:54.483100 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2eeb52e-2438-4d6b-9969-5e4f5a9ad9e0 service nova] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:53:54.508004 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 676/1388] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jun 23 15:53:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 15:53:56.742625 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e78f19-d7d5-4e2b-bce6-813ac7069760 None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:53:56.743315 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e78f19-d7d5-4e2b-bce6-813ac7069760 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:53:56.743315 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e78f19-d7d5-4e2b-bce6-813ac7069760 None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:53:56.744718 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 713/1389] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jun 23 15:53:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 15:53:56.751539 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65702325-b212-48e1-8099-14eef1118cab None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:53:56.752096 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65702325-b212-48e1-8099-14eef1118cab None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:53:56.752096 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65702325-b212-48e1-8099-14eef1118cab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:53:56.771193 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-65702325-b212-48e1-8099-14eef1118cab None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:53:56.772409 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-65702325-b212-48e1-8099-14eef1118cab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:53:56.780787 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 677/1390] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jun 23 15:53:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 15:53:56.825688 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e78f19-d7d5-4e2b-bce6-813ac7069760 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:53:56.840395 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e78f19-d7d5-4e2b-bce6-813ac7069760 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:53:56.845877 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c455123e-dd22-40db-9393-c08751779847 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '705115d19bab424d894bea8b1b48a7f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:53:56.846543 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c455123e-dd22-40db-9393-c08751779847 service nova] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:53:56.846844 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c455123e-dd22-40db-9393-c08751779847 service nova] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:53:56.847179 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c455123e-dd22-40db-9393-c08751779847 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:53:56.856645 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c455123e-dd22-40db-9393-c08751779847 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:53:56.858241 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c455123e-dd22-40db-9393-c08751779847 service nova] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:53:56.868767 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 714/1391] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jun 23 15:53:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 15:53:59.655547 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dfc91f-8ee4-4d49-bc39-e986c81f0ae1 None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:53:59.656111 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dfc91f-8ee4-4d49-bc39-e986c81f0ae1 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:53:59.656218 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dfc91f-8ee4-4d49-bc39-e986c81f0ae1 None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:53:59.657557 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 678/1392] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jun 23 15:53:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 15:53:59.664761 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891d2ad5-6207-4813-ac28-037ff35ef070 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:53:59.665401 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891d2ad5-6207-4813-ac28-037ff35ef070 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:53:59.665401 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891d2ad5-6207-4813-ac28-037ff35ef070 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:53:59.687883 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-891d2ad5-6207-4813-ac28-037ff35ef070 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:53:59.689119 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-891d2ad5-6207-4813-ac28-037ff35ef070 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:53:59.697110 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 715/1393] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jun 23 15:53:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 15:53:59.742837 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65dfc91f-8ee4-4d49-bc39-e986c81f0ae1 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:53:59.751191 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=46 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:53:59.758975 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65dfc91f-8ee4-4d49-bc39-e986c81f0ae1 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:53:59.764471 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea07b62a-827c-4ed0-baaf-559d0642a077 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '705115d19bab424d894bea8b1b48a7f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:53:59.765145 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea07b62a-827c-4ed0-baaf-559d0642a077 service nova] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:53:59.765425 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea07b62a-827c-4ed0-baaf-559d0642a077 service nova] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:53:59.765707 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea07b62a-827c-4ed0-baaf-559d0642a077 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:53:59.775153 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea07b62a-827c-4ed0-baaf-559d0642a077 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:53:59.776770 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea07b62a-827c-4ed0-baaf-559d0642a077 service nova] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:53:59.788333 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 679/1394] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jun 23 15:53:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 15:54:02.035827 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63971390-c474-4ffc-b0a9-040e532afd57 None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:54:02.036403 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63971390-c474-4ffc-b0a9-040e532afd57 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:54:02.036403 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63971390-c474-4ffc-b0a9-040e532afd57 None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:54:02.038587 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 716/1395] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jun 23 15:54:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 15:54:02.044330 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ded6d5-196a-42ea-9ca0-f388330102f2 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:54:02.044828 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ded6d5-196a-42ea-9ca0-f388330102f2 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:54:02.044933 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ded6d5-196a-42ea-9ca0-f388330102f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:54:02.071615 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-c1ded6d5-196a-42ea-9ca0-f388330102f2 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:54:02.073688 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-c1ded6d5-196a-42ea-9ca0-f388330102f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:54:02.088026 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 680/1396] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jun 23 15:54:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 15:54:02.134258 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63971390-c474-4ffc-b0a9-040e532afd57 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:54:02.152504 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63971390-c474-4ffc-b0a9-040e532afd57 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:54:02.158832 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4a656d-1405-4b00-863b-77b31f007c11 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '705115d19bab424d894bea8b1b48a7f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:54:02.159503 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4a656d-1405-4b00-863b-77b31f007c11 service nova] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:54:02.159790 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4a656d-1405-4b00-863b-77b31f007c11 service nova] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:54:02.160071 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4a656d-1405-4b00-863b-77b31f007c11 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:54:02.169880 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4a656d-1405-4b00-863b-77b31f007c11 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:54:02.171536 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4a656d-1405-4b00-863b-77b31f007c11 service nova] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:54:02.184240 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 717/1397] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jun 23 15:54:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 15:54:02.316355 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00ba25b-d527-411d-afe2-836134d51b04 None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:54:02.316744 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00ba25b-d527-411d-afe2-836134d51b04 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:54:02.317027 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00ba25b-d527-411d-afe2-836134d51b04 None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:54:02.318601 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 681/1398] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jun 23 15:54:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 15:54:02.325776 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912ba22b-ce4a-4453-95b3-39e2d5c3cdc3 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:54:02.326050 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912ba22b-ce4a-4453-95b3-39e2d5c3cdc3 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:54:02.326316 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912ba22b-ce4a-4453-95b3-39e2d5c3cdc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:54:02.348056 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-912ba22b-ce4a-4453-95b3-39e2d5c3cdc3 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:54:02.349342 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-912ba22b-ce4a-4453-95b3-39e2d5c3cdc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:54:02.364772 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 718/1399] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jun 23 15:54:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 15:54:02.424183 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8179cc38-e8c3-43ba-b5f4-a4d9dc148a2b None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:54:02.424558 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8179cc38-e8c3-43ba-b5f4-a4d9dc148a2b None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:54:02.424904 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8179cc38-e8c3-43ba-b5f4-a4d9dc148a2b None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:54:02.426349 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 682/1400] 10.222.0.46 () {54 vars in 962 bytes} [Thu Jun 23 15:54:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 15:54:02.431832 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea39a44-1e41-4c8b-b869-aa608d22d571 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:54:02.432126 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea39a44-1e41-4c8b-b869-aa608d22d571 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:54:02.432407 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea39a44-1e41-4c8b-b869-aa608d22d571 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:54:02.486023 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-2ea39a44-1e41-4c8b-b869-aa608d22d571 None None] MFA Rules not processed for user `83970271286e451e9008bbf07cd3fa86`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:54:02.487186 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-2ea39a44-1e41-4c8b-b869-aa608d22d571 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:54:02.526144 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:54:02.574719 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 719/1401] 10.222.0.46 () {58 vars in 1059 bytes} [Thu Jun 23 15:54:02 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 15:54:02.583471 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8179cc38-e8c3-43ba-b5f4-a4d9dc148a2b None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:54:02.596079 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8179cc38-e8c3-43ba-b5f4-a4d9dc148a2b None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:54:02.599823 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6769e961-113c-49d4-9b5b-46c355f09ab9 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83970271286e451e9008bbf07cd3fa86', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:54:02.600533 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6769e961-113c-49d4-9b5b-46c355f09ab9 service glance] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:54:02.600830 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6769e961-113c-49d4-9b5b-46c355f09ab9 service glance] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:54:02.601112 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6769e961-113c-49d4-9b5b-46c355f09ab9 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:54:02.611346 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6769e961-113c-49d4-9b5b-46c355f09ab9 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:54:02.612842 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6769e961-113c-49d4-9b5b-46c355f09ab9 service glance] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:54:02.623703 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 683/1402] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jun 23 15:54:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 15:54:03.392512 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d66db5-f53f-49d9-9402-62f616cf80e8 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:54:03.393031 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d66db5-f53f-49d9-9402-62f616cf80e8 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:54:03.393257 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d66db5-f53f-49d9-9402-62f616cf80e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:54:03.450171 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-d1d66db5-f53f-49d9-9402-62f616cf80e8 None None] MFA Rules not processed for user `be48addfaa154555b8d0cb7aac8b32ac`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:54:03.451542 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-d1d66db5-f53f-49d9-9402-62f616cf80e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:54:03.523182 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 720/1403] 10.222.0.46 () {58 vars in 1009 bytes} [Thu Jun 23 15:54:03 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 15:54:03.577396 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d11093-7cd0-4e6a-a984-6c9f5a5224fa None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:54:03.577968 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d11093-7cd0-4e6a-a984-6c9f5a5224fa None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:54:03.577968 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d11093-7cd0-4e6a-a984-6c9f5a5224fa None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:54:03.579928 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 684/1404] 10.222.0.46 () {54 vars in 942 bytes} [Thu Jun 23 15:54:03 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 15:54:03.586610 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cea03de-9ea0-42ed-9d34-a465276c4ad8 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:54:03.586995 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cea03de-9ea0-42ed-9d34-a465276c4ad8 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:54:03.587353 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cea03de-9ea0-42ed-9d34-a465276c4ad8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:54:03.639045 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-7cea03de-9ea0-42ed-9d34-a465276c4ad8 None None] MFA Rules not processed for user `242bf02d14724fe4837fcce8730377f9`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:54:03.640248 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-7cea03de-9ea0-42ed-9d34-a465276c4ad8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:54:03.717957 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 721/1405] 10.222.0.46 () {58 vars in 1039 bytes} [Thu Jun 23 15:54:03 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 15:54:03.729035 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d11093-7cd0-4e6a-a984-6c9f5a5224fa None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:54:03.746617 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d11093-7cd0-4e6a-a984-6c9f5a5224fa None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:54:03.755053 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9e4b53-b2d3-4e73-86b0-aaff05a052d8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '242bf02d14724fe4837fcce8730377f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:54:03.755723 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9e4b53-b2d3-4e73-86b0-aaff05a052d8 service swift] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:54:03.756014 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9e4b53-b2d3-4e73-86b0-aaff05a052d8 service swift] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:54:03.756324 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9e4b53-b2d3-4e73-86b0-aaff05a052d8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:54:03.765651 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9e4b53-b2d3-4e73-86b0-aaff05a052d8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:54:03.767277 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9e4b53-b2d3-4e73-86b0-aaff05a052d8 service swift] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:54:03.778159 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 685/1406] 10.222.0.46 () {60 vars in 1384 bytes} [Thu Jun 23 15:54:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 23 15:54:04.324371 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae0e5e0-0e72-4bee-8957-6903962531ee None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:54:04.324788 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae0e5e0-0e72-4bee-8957-6903962531ee None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:54:04.324945 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae0e5e0-0e72-4bee-8957-6903962531ee None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:54:04.326340 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 722/1407] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jun 23 15:54:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 15:54:04.331953 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cc0281-2aac-43f3-b600-98f3a87d722e None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:54:04.332247 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cc0281-2aac-43f3-b600-98f3a87d722e None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:54:04.332528 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cc0281-2aac-43f3-b600-98f3a87d722e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:54:04.352125 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-23cc0281-2aac-43f3-b600-98f3a87d722e None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:54:04.353363 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-23cc0281-2aac-43f3-b600-98f3a87d722e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:54:04.362493 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 686/1408] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jun 23 15:54:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 15:54:04.409435 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae0e5e0-0e72-4bee-8957-6903962531ee None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:54:04.431482 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae0e5e0-0e72-4bee-8957-6903962531ee None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:54:04.437268 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc8bff0-3e07-418b-838d-7d81f3c82680 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '705115d19bab424d894bea8b1b48a7f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:54:04.437939 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc8bff0-3e07-418b-838d-7d81f3c82680 service nova] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:54:04.438364 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc8bff0-3e07-418b-838d-7d81f3c82680 service nova] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:54:04.438541 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc8bff0-3e07-418b-838d-7d81f3c82680 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:54:04.450707 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc8bff0-3e07-418b-838d-7d81f3c82680 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:54:04.452435 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc8bff0-3e07-418b-838d-7d81f3c82680 service nova] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:54:04.458769 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:54:04.465774 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 723/1409] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jun 23 15:54:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 15:54:14.346096 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=23 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:54:27.231277 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d23450-a2d1-44e1-accd-daad6fcfc5a2 None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:54:27.231944 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d23450-a2d1-44e1-accd-daad6fcfc5a2 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:54:27.231944 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d23450-a2d1-44e1-accd-daad6fcfc5a2 None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:54:27.233206 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 687/1410] 10.222.0.46 () {54 vars in 918 bytes} [Thu Jun 23 15:54:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 15:54:27.238249 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4540ea-8a98-47ff-a00f-4107e771f667 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:54:27.239024 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4540ea-8a98-47ff-a00f-4107e771f667 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:54:27.239024 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4540ea-8a98-47ff-a00f-4107e771f667 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:54:27.257131 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-5a4540ea-8a98-47ff-a00f-4107e771f667 None None] MFA Rules not processed for user `3f66988b73644fe38b9849663ee70f25`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:54:27.258320 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-5a4540ea-8a98-47ff-a00f-4107e771f667 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:54:27.266631 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 724/1411] 10.222.0.46 () {58 vars in 1015 bytes} [Thu Jun 23 15:54:27 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 15:54:27.281418 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87e04c4-3f70-4d6a-9b23-b69549cdf952 None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:54:27.281713 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87e04c4-3f70-4d6a-9b23-b69549cdf952 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:54:27.281996 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87e04c4-3f70-4d6a-9b23-b69549cdf952 None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:54:27.283406 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 688/1412] 10.222.0.46 () {54 vars in 918 bytes} [Thu Jun 23 15:54:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 15:54:27.288671 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdb42a0-e629-4f63-bf23-d055be2203a0 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:54:27.288963 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdb42a0-e629-4f63-bf23-d055be2203a0 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:54:27.289242 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdb42a0-e629-4f63-bf23-d055be2203a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:54:27.309576 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-7cdb42a0-e629-4f63-bf23-d055be2203a0 None None] MFA Rules not processed for user `705115d19bab424d894bea8b1b48a7f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:54:27.310840 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-7cdb42a0-e629-4f63-bf23-d055be2203a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:54:27.320295 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 725/1413] 10.222.0.46 () {58 vars in 1015 bytes} [Thu Jun 23 15:54:27 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 15:54:29.859334 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4521ae8a-e021-446f-a82f-72b644f89c63 None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:54:29.860000 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4521ae8a-e021-446f-a82f-72b644f89c63 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:54:29.860000 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4521ae8a-e021-446f-a82f-72b644f89c63 None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:54:29.861313 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 689/1414] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 23 15:54:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 15:54:29.867707 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9b02ec-df84-4f78-bf27-2b7180396d6f None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:54:29.868213 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9b02ec-df84-4f78-bf27-2b7180396d6f None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:54:29.868316 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9b02ec-df84-4f78-bf27-2b7180396d6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:54:29.887092 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-ca9b02ec-df84-4f78-bf27-2b7180396d6f None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:54:29.888324 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-ca9b02ec-df84-4f78-bf27-2b7180396d6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:54:29.896431 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 726/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 23 15:54:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 15:54:29.941395 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4521ae8a-e021-446f-a82f-72b644f89c63 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:54:29.956576 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4521ae8a-e021-446f-a82f-72b644f89c63 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:54:29.962008 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e77c55-1b33-4730-b656-29f7a38e76ea service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '705115d19bab424d894bea8b1b48a7f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:54:29.962666 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e77c55-1b33-4730-b656-29f7a38e76ea service nova] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:54:29.962979 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e77c55-1b33-4730-b656-29f7a38e76ea service nova] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:54:29.963253 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e77c55-1b33-4730-b656-29f7a38e76ea service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:54:29.972408 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e77c55-1b33-4730-b656-29f7a38e76ea service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:54:29.974061 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e77c55-1b33-4730-b656-29f7a38e76ea service nova] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:54:29.984301 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 690/1416] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jun 23 15:54:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 15:54:30.928880 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f97eae1-2ba6-40e4-9c42-28702d31775c None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:54:30.929459 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f97eae1-2ba6-40e4-9c42-28702d31775c None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:54:30.929459 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f97eae1-2ba6-40e4-9c42-28702d31775c None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:54:30.930842 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 727/1417] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 23 15:54:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 15:54:30.936127 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c809427-950e-4e9c-afd2-819b9e81ea62 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:54:30.936530 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c809427-950e-4e9c-afd2-819b9e81ea62 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:54:30.936685 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c809427-950e-4e9c-afd2-819b9e81ea62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:54:30.955822 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-7c809427-950e-4e9c-afd2-819b9e81ea62 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:54:30.957029 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-7c809427-950e-4e9c-afd2-819b9e81ea62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:54:30.965044 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 691/1418] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 23 15:54:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 15:54:31.118270 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f97eae1-2ba6-40e4-9c42-28702d31775c None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:54:31.137706 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f97eae1-2ba6-40e4-9c42-28702d31775c None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:54:31.146038 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c735a56-0fdb-4ec0-9ded-ccc3e308e8c8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '705115d19bab424d894bea8b1b48a7f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:54:31.147567 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c735a56-0fdb-4ec0-9ded-ccc3e308e8c8 service nova] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:54:31.147864 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c735a56-0fdb-4ec0-9ded-ccc3e308e8c8 service nova] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:54:31.148147 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c735a56-0fdb-4ec0-9ded-ccc3e308e8c8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:54:31.156902 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c735a56-0fdb-4ec0-9ded-ccc3e308e8c8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:54:31.158506 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c735a56-0fdb-4ec0-9ded-ccc3e308e8c8 service nova] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:54:31.168708 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 728/1419] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jun 23 15:54:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 15:54:32.061363 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad365ce-abcd-4a2c-8eb1-6ea138e0c7fd None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:54:32.062299 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad365ce-abcd-4a2c-8eb1-6ea138e0c7fd None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:54:32.066057 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad365ce-abcd-4a2c-8eb1-6ea138e0c7fd None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:54:32.066057 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 692/1420] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 23 15:54:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 15:54:32.069715 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e2cac0-8ff8-45d8-b385-4d6a192220ad None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:54:32.070190 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e2cac0-8ff8-45d8-b385-4d6a192220ad None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:54:32.070294 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e2cac0-8ff8-45d8-b385-4d6a192220ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:54:32.088217 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-b1e2cac0-8ff8-45d8-b385-4d6a192220ad None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:54:32.089414 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-b1e2cac0-8ff8-45d8-b385-4d6a192220ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:54:32.097229 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 729/1421] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 23 15:54:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 15:54:32.249889 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad365ce-abcd-4a2c-8eb1-6ea138e0c7fd None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:54:32.264259 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad365ce-abcd-4a2c-8eb1-6ea138e0c7fd None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:54:32.269560 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd6ef23-88c1-402b-8cd4-956df0ef3a40 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '705115d19bab424d894bea8b1b48a7f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:54:32.270205 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd6ef23-88c1-402b-8cd4-956df0ef3a40 service nova] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:54:32.270491 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd6ef23-88c1-402b-8cd4-956df0ef3a40 service nova] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:54:32.270820 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd6ef23-88c1-402b-8cd4-956df0ef3a40 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:54:32.279954 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd6ef23-88c1-402b-8cd4-956df0ef3a40 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:54:32.281542 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd6ef23-88c1-402b-8cd4-956df0ef3a40 service nova] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:54:32.291645 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 693/1422] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jun 23 15:54:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 15:54:33.178683 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdaf413-8f62-467e-9f9e-882ca40237e0 None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:54:33.179268 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdaf413-8f62-467e-9f9e-882ca40237e0 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:54:33.179268 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdaf413-8f62-467e-9f9e-882ca40237e0 None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:54:33.180652 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 730/1423] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 23 15:54:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 15:54:33.185768 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aada683-a79b-4c4b-a518-c1ca98fe6136 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:54:33.186190 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aada683-a79b-4c4b-a518-c1ca98fe6136 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:54:33.186337 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aada683-a79b-4c4b-a518-c1ca98fe6136 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:54:33.204629 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-7aada683-a79b-4c4b-a518-c1ca98fe6136 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:54:33.205840 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-7aada683-a79b-4c4b-a518-c1ca98fe6136 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:54:33.213838 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 694/1424] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 23 15:54:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 15:54:33.365394 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdaf413-8f62-467e-9f9e-882ca40237e0 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:54:33.379686 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdaf413-8f62-467e-9f9e-882ca40237e0 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:54:33.385642 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d11d3c9-52a8-40f6-916d-2d3421ebab7d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '705115d19bab424d894bea8b1b48a7f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:54:33.386286 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d11d3c9-52a8-40f6-916d-2d3421ebab7d service nova] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:54:33.386572 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d11d3c9-52a8-40f6-916d-2d3421ebab7d service nova] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:54:33.386862 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d11d3c9-52a8-40f6-916d-2d3421ebab7d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:54:33.395108 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d11d3c9-52a8-40f6-916d-2d3421ebab7d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:54:33.396963 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d11d3c9-52a8-40f6-916d-2d3421ebab7d service nova] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:54:33.407023 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 731/1425] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jun 23 15:54:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 15:54:34.268619 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a72b50-f509-4e89-a3e2-b35c39188bf8 None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:54:34.269195 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a72b50-f509-4e89-a3e2-b35c39188bf8 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:54:34.269195 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a72b50-f509-4e89-a3e2-b35c39188bf8 None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:54:34.270548 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 695/1426] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 23 15:54:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 15:54:34.275860 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2628ab21-741e-445b-a2fd-eec696192be9 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:54:34.276623 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2628ab21-741e-445b-a2fd-eec696192be9 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:54:34.276908 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2628ab21-741e-445b-a2fd-eec696192be9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:54:34.295201 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-2628ab21-741e-445b-a2fd-eec696192be9 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:54:34.296405 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-2628ab21-741e-445b-a2fd-eec696192be9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:54:34.306270 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 732/1427] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 23 15:54:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 15:54:34.461342 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a72b50-f509-4e89-a3e2-b35c39188bf8 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:54:34.477626 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a72b50-f509-4e89-a3e2-b35c39188bf8 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:54:34.483328 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bb3b5f-aa89-44b2-8eb3-9ac49bc195df service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '705115d19bab424d894bea8b1b48a7f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:54:34.483978 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bb3b5f-aa89-44b2-8eb3-9ac49bc195df service nova] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:54:34.484264 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bb3b5f-aa89-44b2-8eb3-9ac49bc195df service nova] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:54:34.484537 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bb3b5f-aa89-44b2-8eb3-9ac49bc195df service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:54:34.494618 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4bb3b5f-aa89-44b2-8eb3-9ac49bc195df service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:54:34.496248 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4bb3b5f-aa89-44b2-8eb3-9ac49bc195df service nova] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:54:34.506600 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 696/1428] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jun 23 15:54:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 15:54:35.311498 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41feb6a4-66c6-4389-b1f9-4d321d6730ea None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:54:35.312103 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41feb6a4-66c6-4389-b1f9-4d321d6730ea None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:54:35.312103 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41feb6a4-66c6-4389-b1f9-4d321d6730ea None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:54:35.313585 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 733/1429] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 23 15:54:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 15:54:35.318780 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9513b8b0-2cc6-4e4a-902d-c963741c379d None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:54:35.319202 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9513b8b0-2cc6-4e4a-902d-c963741c379d None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:54:35.319341 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9513b8b0-2cc6-4e4a-902d-c963741c379d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:54:35.337637 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-9513b8b0-2cc6-4e4a-902d-c963741c379d None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:54:35.338879 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-9513b8b0-2cc6-4e4a-902d-c963741c379d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:54:35.346734 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 697/1430] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 23 15:54:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 15:54:35.495241 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41feb6a4-66c6-4389-b1f9-4d321d6730ea None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:54:35.509924 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41feb6a4-66c6-4389-b1f9-4d321d6730ea None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:54:35.515513 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b45cafd-6611-4383-bcd4-a7a18614d712 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '705115d19bab424d894bea8b1b48a7f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:54:35.516145 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b45cafd-6611-4383-bcd4-a7a18614d712 service nova] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:54:35.516421 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b45cafd-6611-4383-bcd4-a7a18614d712 service nova] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:54:35.516696 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b45cafd-6611-4383-bcd4-a7a18614d712 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:54:35.525735 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b45cafd-6611-4383-bcd4-a7a18614d712 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:54:35.527365 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b45cafd-6611-4383-bcd4-a7a18614d712 service nova] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:54:35.537258 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 734/1431] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jun 23 15:54:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 15:54:36.348203 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0adcea5-4e1f-4567-8399-90019e261f55 None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:54:36.348751 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0adcea5-4e1f-4567-8399-90019e261f55 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:54:36.348751 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0adcea5-4e1f-4567-8399-90019e261f55 None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:54:36.350738 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 698/1432] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 23 15:54:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 15:54:36.355948 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aeafa28-b856-45e0-9aee-d0faacc763be None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:54:36.356412 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aeafa28-b856-45e0-9aee-d0faacc763be None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:54:36.356514 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aeafa28-b856-45e0-9aee-d0faacc763be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:54:36.374945 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-9aeafa28-b856-45e0-9aee-d0faacc763be None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:54:36.376129 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-9aeafa28-b856-45e0-9aee-d0faacc763be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:54:36.383851 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 735/1433] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 23 15:54:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 15:54:36.531766 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0adcea5-4e1f-4567-8399-90019e261f55 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:54:36.546888 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0adcea5-4e1f-4567-8399-90019e261f55 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:54:36.552354 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba773b1-741b-4d68-aef0-ddd8cc534a2d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '705115d19bab424d894bea8b1b48a7f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:54:36.553010 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba773b1-741b-4d68-aef0-ddd8cc534a2d service nova] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:54:36.553890 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba773b1-741b-4d68-aef0-ddd8cc534a2d service nova] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:54:36.554169 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba773b1-741b-4d68-aef0-ddd8cc534a2d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:54:36.563060 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba773b1-741b-4d68-aef0-ddd8cc534a2d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:54:36.564738 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba773b1-741b-4d68-aef0-ddd8cc534a2d service nova] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:54:36.574639 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 699/1434] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jun 23 15:54:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 15:54:37.387610 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8562c02-a038-4c6a-a2e0-4e306f24af1e None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:54:37.388278 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8562c02-a038-4c6a-a2e0-4e306f24af1e None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:54:37.388278 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8562c02-a038-4c6a-a2e0-4e306f24af1e None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:54:37.389560 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 736/1435] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 23 15:54:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 15:54:37.394772 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92450d73-3c5e-4879-9547-112c34512743 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:54:37.395266 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92450d73-3c5e-4879-9547-112c34512743 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:54:37.395368 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92450d73-3c5e-4879-9547-112c34512743 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:54:37.415201 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-92450d73-3c5e-4879-9547-112c34512743 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:54:37.416430 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-92450d73-3c5e-4879-9547-112c34512743 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:54:37.424365 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 700/1436] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 23 15:54:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 15:54:37.574892 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8562c02-a038-4c6a-a2e0-4e306f24af1e None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:54:37.589713 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8562c02-a038-4c6a-a2e0-4e306f24af1e None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:54:37.595106 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2be7c9-8f40-4172-92c7-89cd896d1803 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '705115d19bab424d894bea8b1b48a7f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:54:37.595748 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2be7c9-8f40-4172-92c7-89cd896d1803 service nova] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:54:37.596034 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2be7c9-8f40-4172-92c7-89cd896d1803 service nova] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:54:37.596311 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2be7c9-8f40-4172-92c7-89cd896d1803 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:54:37.605129 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2be7c9-8f40-4172-92c7-89cd896d1803 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:54:37.606747 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2be7c9-8f40-4172-92c7-89cd896d1803 service nova] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:54:37.616078 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 737/1437] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jun 23 15:54:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 15:54:38.412323 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2517988a-6b4a-4366-8850-d7cd99f74119 None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:54:38.412890 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2517988a-6b4a-4366-8850-d7cd99f74119 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:54:38.412890 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2517988a-6b4a-4366-8850-d7cd99f74119 None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:54:38.414264 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 701/1438] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 23 15:54:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 15:54:38.419332 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1734cad4-781b-48ca-8260-1dadba0aeff8 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:54:38.419769 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1734cad4-781b-48ca-8260-1dadba0aeff8 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:54:38.419898 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1734cad4-781b-48ca-8260-1dadba0aeff8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:54:38.438463 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-1734cad4-781b-48ca-8260-1dadba0aeff8 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:54:38.439714 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-1734cad4-781b-48ca-8260-1dadba0aeff8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:54:38.448371 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 738/1439] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 23 15:54:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 15:54:38.600713 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2517988a-6b4a-4366-8850-d7cd99f74119 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:54:38.614842 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2517988a-6b4a-4366-8850-d7cd99f74119 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:54:38.620794 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c349760c-a932-4760-9ce3-02ce401abb30 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '705115d19bab424d894bea8b1b48a7f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:54:38.621435 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c349760c-a932-4760-9ce3-02ce401abb30 service nova] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:54:38.621720 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c349760c-a932-4760-9ce3-02ce401abb30 service nova] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:54:38.621998 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c349760c-a932-4760-9ce3-02ce401abb30 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:54:38.630157 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c349760c-a932-4760-9ce3-02ce401abb30 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:54:38.631816 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c349760c-a932-4760-9ce3-02ce401abb30 service nova] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:54:38.641232 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 702/1440] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jun 23 15:54:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 15:54:40.717240 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43337269-46d6-4b47-a27c-593e19042fa2 None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:54:40.717830 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43337269-46d6-4b47-a27c-593e19042fa2 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:54:40.717830 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43337269-46d6-4b47-a27c-593e19042fa2 None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:54:40.719489 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 739/1441] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 23 15:54:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 15:54:40.726354 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6bb220-662a-4173-aa63-012c50a13161 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:54:40.726914 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6bb220-662a-4173-aa63-012c50a13161 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:54:40.726914 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6bb220-662a-4173-aa63-012c50a13161 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:54:40.745476 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-de6bb220-662a-4173-aa63-012c50a13161 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:54:40.746729 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-de6bb220-662a-4173-aa63-012c50a13161 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:54:40.754944 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 703/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 23 15:54:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 15:54:40.824856 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43337269-46d6-4b47-a27c-593e19042fa2 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:54:40.826713 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-43337269-46d6-4b47-a27c-593e19042fa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:54:40.845220 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43337269-46d6-4b47-a27c-593e19042fa2 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:54:40.852629 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187b4690-3d27-4fbc-b150-82367cb8272d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51784064951a4619ae09b9167fb0fd0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:54:40.853307 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187b4690-3d27-4fbc-b150-82367cb8272d service neutron] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:54:40.853589 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187b4690-3d27-4fbc-b150-82367cb8272d service neutron] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:54:40.853859 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187b4690-3d27-4fbc-b150-82367cb8272d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:54:40.867376 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187b4690-3d27-4fbc-b150-82367cb8272d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:54:40.870897 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187b4690-3d27-4fbc-b150-82367cb8272d service neutron] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:54:40.890044 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 740/1443] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jun 23 15:54:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 15:54:43.322987 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76783dc4-c852-48ce-8486-5d5e91a937da None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:54:43.323642 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76783dc4-c852-48ce-8486-5d5e91a937da None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:54:43.323642 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76783dc4-c852-48ce-8486-5d5e91a937da None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:54:43.324968 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 704/1444] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 23 15:54:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 15:54:43.331109 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817ed807-e4da-4d29-8f9a-024f0eb6ab2d None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:54:43.331641 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817ed807-e4da-4d29-8f9a-024f0eb6ab2d None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:54:43.331739 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817ed807-e4da-4d29-8f9a-024f0eb6ab2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:54:43.351577 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-817ed807-e4da-4d29-8f9a-024f0eb6ab2d None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:54:43.352820 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-817ed807-e4da-4d29-8f9a-024f0eb6ab2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:54:43.360929 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 741/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 23 15:54:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 15:54:43.403368 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76783dc4-c852-48ce-8486-5d5e91a937da None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:54:43.415383 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76783dc4-c852-48ce-8486-5d5e91a937da None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:54:43.418818 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd159e51-938b-4988-b1dd-bb75db96be0a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51784064951a4619ae09b9167fb0fd0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:54:43.419477 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd159e51-938b-4988-b1dd-bb75db96be0a service neutron] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:54:43.419758 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd159e51-938b-4988-b1dd-bb75db96be0a service neutron] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:54:43.420047 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd159e51-938b-4988-b1dd-bb75db96be0a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:54:43.430103 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd159e51-938b-4988-b1dd-bb75db96be0a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:54:43.431609 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd159e51-938b-4988-b1dd-bb75db96be0a service neutron] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:54:43.441971 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 705/1446] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jun 23 15:54:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 15:54:46.081097 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63af687-4882-40a2-b8e1-1998b49617c1 None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:54:46.081686 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63af687-4882-40a2-b8e1-1998b49617c1 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:54:46.082004 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63af687-4882-40a2-b8e1-1998b49617c1 None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:54:46.084395 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 742/1447] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 23 15:54:46 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 15:54:46.092854 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2296ba67-bcf6-402c-89f8-dae5f0d5c911 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:54:46.093406 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2296ba67-bcf6-402c-89f8-dae5f0d5c911 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:54:46.093868 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2296ba67-bcf6-402c-89f8-dae5f0d5c911 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:54:46.112559 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-2296ba67-bcf6-402c-89f8-dae5f0d5c911 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:54:46.113769 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-2296ba67-bcf6-402c-89f8-dae5f0d5c911 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:54:46.122457 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 706/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 23 15:54:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 15:54:46.170204 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63af687-4882-40a2-b8e1-1998b49617c1 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:54:46.181594 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63af687-4882-40a2-b8e1-1998b49617c1 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:54:46.185437 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab392797-7b6a-444f-bd8b-0feb518a9dc9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51784064951a4619ae09b9167fb0fd0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:54:46.186089 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab392797-7b6a-444f-bd8b-0feb518a9dc9 service neutron] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:54:46.186374 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab392797-7b6a-444f-bd8b-0feb518a9dc9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:54:46.186652 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab392797-7b6a-444f-bd8b-0feb518a9dc9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:54:46.197395 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab392797-7b6a-444f-bd8b-0feb518a9dc9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:54:46.199063 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab392797-7b6a-444f-bd8b-0feb518a9dc9 service neutron] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:54:46.210304 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 743/1449] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jun 23 15:54:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 15:54:48.858725 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc80d54-fb03-4956-a6b0-e4bd0108e1f4 None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:54:48.859541 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc80d54-fb03-4956-a6b0-e4bd0108e1f4 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:54:48.859732 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc80d54-fb03-4956-a6b0-e4bd0108e1f4 None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:54:48.861337 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 707/1450] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 23 15:54:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 15:54:48.868566 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e006670-2b5f-4678-817d-dcb9f05753d1 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:54:48.869276 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e006670-2b5f-4678-817d-dcb9f05753d1 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:54:48.869589 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e006670-2b5f-4678-817d-dcb9f05753d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:54:48.899804 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-8e006670-2b5f-4678-817d-dcb9f05753d1 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:54:48.901979 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-8e006670-2b5f-4678-817d-dcb9f05753d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:54:48.914522 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 744/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 23 15:54:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 15:54:48.953503 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc80d54-fb03-4956-a6b0-e4bd0108e1f4 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:54:48.965136 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc80d54-fb03-4956-a6b0-e4bd0108e1f4 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:54:48.969106 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d286bf-b690-4205-9d02-9745bbf86763 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51784064951a4619ae09b9167fb0fd0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:54:48.969869 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d286bf-b690-4205-9d02-9745bbf86763 service neutron] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:54:48.970201 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d286bf-b690-4205-9d02-9745bbf86763 service neutron] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:54:48.970571 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d286bf-b690-4205-9d02-9745bbf86763 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:54:48.981648 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d286bf-b690-4205-9d02-9745bbf86763 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:54:48.983160 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d286bf-b690-4205-9d02-9745bbf86763 service neutron] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:54:48.995376 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 708/1452] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jun 23 15:54:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 15:54:51.168722 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bbf260-d878-4982-a18a-4c5336660eff None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:54:51.169260 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bbf260-d878-4982-a18a-4c5336660eff None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:54:51.169260 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bbf260-d878-4982-a18a-4c5336660eff None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:54:51.170661 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 745/1453] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 23 15:54:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 15:54:51.175912 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd60275b-f8cb-4e95-86d0-566a8aa5ec1c None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:54:51.176333 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd60275b-f8cb-4e95-86d0-566a8aa5ec1c None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:54:51.176473 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd60275b-f8cb-4e95-86d0-566a8aa5ec1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:54:51.194821 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-fd60275b-f8cb-4e95-86d0-566a8aa5ec1c None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:54:51.196027 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-fd60275b-f8cb-4e95-86d0-566a8aa5ec1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:54:51.203808 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 709/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 23 15:54:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 15:54:51.242315 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bbf260-d878-4982-a18a-4c5336660eff None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:54:51.253557 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bbf260-d878-4982-a18a-4c5336660eff None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:54:51.257094 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f84522-8532-4918-90e4-479fe366eba6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51784064951a4619ae09b9167fb0fd0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:54:51.257727 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f84522-8532-4918-90e4-479fe366eba6 service neutron] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:54:51.258008 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f84522-8532-4918-90e4-479fe366eba6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:54:51.258284 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f84522-8532-4918-90e4-479fe366eba6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:54:51.267170 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f84522-8532-4918-90e4-479fe366eba6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:54:51.268610 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f84522-8532-4918-90e4-479fe366eba6 service neutron] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:54:51.278273 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 746/1455] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jun 23 15:54:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 15:54:53.364855 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90701f0-420b-4eb4-a420-7a52075bcda2 None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:54:53.365439 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90701f0-420b-4eb4-a420-7a52075bcda2 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:54:53.365439 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90701f0-420b-4eb4-a420-7a52075bcda2 None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:54:53.366829 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 710/1456] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 23 15:54:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 15:54:53.372028 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df1d368-c4ba-41d2-be22-f5493fcd3da6 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:54:53.372451 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df1d368-c4ba-41d2-be22-f5493fcd3da6 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:54:53.372594 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df1d368-c4ba-41d2-be22-f5493fcd3da6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:54:53.390406 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-0df1d368-c4ba-41d2-be22-f5493fcd3da6 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:54:53.391714 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-0df1d368-c4ba-41d2-be22-f5493fcd3da6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:54:53.400265 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 747/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 23 15:54:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 15:54:53.437839 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90701f0-420b-4eb4-a420-7a52075bcda2 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:54:53.448788 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90701f0-420b-4eb4-a420-7a52075bcda2 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:54:53.452344 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84eec455-9aab-49ae-9615-2f37c6d0661d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51784064951a4619ae09b9167fb0fd0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:54:53.452989 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84eec455-9aab-49ae-9615-2f37c6d0661d service neutron] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:54:53.453271 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84eec455-9aab-49ae-9615-2f37c6d0661d service neutron] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:54:53.453551 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84eec455-9aab-49ae-9615-2f37c6d0661d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:54:53.462992 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84eec455-9aab-49ae-9615-2f37c6d0661d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:54:53.464416 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84eec455-9aab-49ae-9615-2f37c6d0661d service neutron] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:54:53.473602 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 711/1458] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jun 23 15:54:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 15:54:55.617633 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bd768c-5061-4085-b264-17df78f7d9c4 None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:54:55.618186 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bd768c-5061-4085-b264-17df78f7d9c4 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:54:55.618186 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bd768c-5061-4085-b264-17df78f7d9c4 None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:54:55.619643 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 748/1459] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 23 15:54:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 15:54:55.625063 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2874acb0-23fa-4b45-9d30-92d620521138 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:54:55.625554 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2874acb0-23fa-4b45-9d30-92d620521138 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:54:55.625653 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2874acb0-23fa-4b45-9d30-92d620521138 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:54:55.632566 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=70 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:54:55.645590 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-2874acb0-23fa-4b45-9d30-92d620521138 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:54:55.646895 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-2874acb0-23fa-4b45-9d30-92d620521138 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:54:55.655310 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 712/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 23 15:54:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 15:54:55.693729 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bd768c-5061-4085-b264-17df78f7d9c4 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:54:55.699898 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=73 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:54:55.706019 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bd768c-5061-4085-b264-17df78f7d9c4 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:54:55.709679 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7fd864-c23f-4c3d-bd9b-9710d915007f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51784064951a4619ae09b9167fb0fd0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:54:55.710327 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7fd864-c23f-4c3d-bd9b-9710d915007f service neutron] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:54:55.710610 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7fd864-c23f-4c3d-bd9b-9710d915007f service neutron] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:54:55.710917 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7fd864-c23f-4c3d-bd9b-9710d915007f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:54:55.720457 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7fd864-c23f-4c3d-bd9b-9710d915007f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:54:55.721869 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7fd864-c23f-4c3d-bd9b-9710d915007f service neutron] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:54:55.732321 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 749/1461] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jun 23 15:54:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 15:54:57.868190 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29859f3-6a8c-4759-9f7f-e48ff60f93cb None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:54:57.868758 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29859f3-6a8c-4759-9f7f-e48ff60f93cb None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:54:57.868758 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29859f3-6a8c-4759-9f7f-e48ff60f93cb None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:54:57.870127 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 713/1462] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 23 15:54:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 15:54:57.875404 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a94a26-a02e-40f7-aa40-d876d017e166 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:54:57.875819 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a94a26-a02e-40f7-aa40-d876d017e166 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:54:57.875959 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a94a26-a02e-40f7-aa40-d876d017e166 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:54:57.893380 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-88a94a26-a02e-40f7-aa40-d876d017e166 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:54:57.894561 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-88a94a26-a02e-40f7-aa40-d876d017e166 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:54:57.903149 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 750/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 23 15:54:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 15:54:57.942151 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29859f3-6a8c-4759-9f7f-e48ff60f93cb None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:54:57.952984 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29859f3-6a8c-4759-9f7f-e48ff60f93cb None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:54:57.956796 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae936ff1-cdb9-462e-a8c2-93a4467818b2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51784064951a4619ae09b9167fb0fd0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:54:57.957446 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae936ff1-cdb9-462e-a8c2-93a4467818b2 service neutron] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:54:57.957733 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae936ff1-cdb9-462e-a8c2-93a4467818b2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:54:57.958015 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae936ff1-cdb9-462e-a8c2-93a4467818b2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:54:57.967629 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae936ff1-cdb9-462e-a8c2-93a4467818b2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:54:57.969067 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae936ff1-cdb9-462e-a8c2-93a4467818b2 service neutron] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:54:57.979510 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 714/1464] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jun 23 15:54:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 15:55:00.139378 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38841c1-cf31-44fe-b945-084b0d4b3435 None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:55:00.139974 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38841c1-cf31-44fe-b945-084b0d4b3435 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:55:00.139974 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38841c1-cf31-44fe-b945-084b0d4b3435 None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:55:00.141474 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 751/1465] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 23 15:55:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 15:55:00.147881 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c578ba0d-827c-43a7-a527-98f940f84e57 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:55:00.148378 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c578ba0d-827c-43a7-a527-98f940f84e57 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:55:00.148494 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c578ba0d-827c-43a7-a527-98f940f84e57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:55:00.166098 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-c578ba0d-827c-43a7-a527-98f940f84e57 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:55:00.167362 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-c578ba0d-827c-43a7-a527-98f940f84e57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:55:00.175276 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 715/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 23 15:55:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 15:55:00.212827 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38841c1-cf31-44fe-b945-084b0d4b3435 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:55:00.223768 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38841c1-cf31-44fe-b945-084b0d4b3435 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:55:00.236880 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d48a63-268b-4804-ad54-7a1e3047d86b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51784064951a4619ae09b9167fb0fd0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:55:00.237546 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d48a63-268b-4804-ad54-7a1e3047d86b service neutron] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:55:00.237828 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d48a63-268b-4804-ad54-7a1e3047d86b service neutron] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:55:00.238102 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d48a63-268b-4804-ad54-7a1e3047d86b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:55:00.249482 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d48a63-268b-4804-ad54-7a1e3047d86b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:55:00.250966 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d48a63-268b-4804-ad54-7a1e3047d86b service neutron] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:55:00.260844 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 752/1467] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jun 23 15:55:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 15:55:02.363364 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e51bdf2-bde1-4201-b8e1-c6b7e7984f9d None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:55:02.364086 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e51bdf2-bde1-4201-b8e1-c6b7e7984f9d None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:55:02.364086 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e51bdf2-bde1-4201-b8e1-c6b7e7984f9d None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:55:02.365322 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 716/1468] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 23 15:55:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 15:55:02.371425 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10efbacf-6d75-427a-8dda-6be3191571d9 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:55:02.372661 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10efbacf-6d75-427a-8dda-6be3191571d9 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:55:02.372661 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10efbacf-6d75-427a-8dda-6be3191571d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:55:02.394783 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-10efbacf-6d75-427a-8dda-6be3191571d9 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:55:02.396023 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-10efbacf-6d75-427a-8dda-6be3191571d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:55:02.407407 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 753/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 23 15:55:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 15:55:02.446873 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e51bdf2-bde1-4201-b8e1-c6b7e7984f9d None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:55:02.457966 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e51bdf2-bde1-4201-b8e1-c6b7e7984f9d None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:55:02.462067 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20f83d2-6a97-497a-b635-bcb6bed58e46 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51784064951a4619ae09b9167fb0fd0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:55:02.462743 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20f83d2-6a97-497a-b635-bcb6bed58e46 service neutron] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:55:02.463030 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20f83d2-6a97-497a-b635-bcb6bed58e46 service neutron] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:55:02.463308 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20f83d2-6a97-497a-b635-bcb6bed58e46 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:55:02.472526 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20f83d2-6a97-497a-b635-bcb6bed58e46 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:55:02.474015 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20f83d2-6a97-497a-b635-bcb6bed58e46 service neutron] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:55:02.484964 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 717/1470] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jun 23 15:55:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 15:55:12.388155 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=17 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:55:12.480366 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=17 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:55:22.906249 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494b2dbd-eaa1-45f5-ba34-f78228b9b56b None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:55:22.906901 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494b2dbd-eaa1-45f5-ba34-f78228b9b56b None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:55:22.906901 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494b2dbd-eaa1-45f5-ba34-f78228b9b56b None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:55:22.908204 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 754/1471] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jun 23 15:55:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 15:55:22.915927 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9285d5f9-a260-4db2-a7bf-6afaebcd7914 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:55:22.916428 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9285d5f9-a260-4db2-a7bf-6afaebcd7914 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:55:22.916531 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9285d5f9-a260-4db2-a7bf-6afaebcd7914 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:55:22.934736 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-9285d5f9-a260-4db2-a7bf-6afaebcd7914 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:55:22.936285 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-9285d5f9-a260-4db2-a7bf-6afaebcd7914 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:55:22.945475 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 718/1472] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jun 23 15:55:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 15:55:22.997991 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f2e240-087a-4223-bd3f-d6df7ecef9bc None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:55:22.998284 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f2e240-087a-4223-bd3f-d6df7ecef9bc None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:55:22.998613 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f2e240-087a-4223-bd3f-d6df7ecef9bc None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:55:22.999998 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 755/1473] 10.222.0.46 () {54 vars in 962 bytes} [Thu Jun 23 15:55:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 15:55:23.007624 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4868d2-0e77-4b8a-94a7-71dc796edf01 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:55:23.007916 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4868d2-0e77-4b8a-94a7-71dc796edf01 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:55:23.008200 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4868d2-0e77-4b8a-94a7-71dc796edf01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:55:23.026975 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-fa4868d2-0e77-4b8a-94a7-71dc796edf01 None None] MFA Rules not processed for user `83970271286e451e9008bbf07cd3fa86`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:55:23.028660 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-fa4868d2-0e77-4b8a-94a7-71dc796edf01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:55:23.036464 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 719/1474] 10.222.0.46 () {58 vars in 1059 bytes} [Thu Jun 23 15:55:23 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 15:55:23.044999 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f2e240-087a-4223-bd3f-d6df7ecef9bc None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:55:23.059107 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f2e240-087a-4223-bd3f-d6df7ecef9bc None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:55:23.062828 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513777ba-d13f-4f63-be25-1a39f6edd6fa service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83970271286e451e9008bbf07cd3fa86', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:55:23.063487 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513777ba-d13f-4f63-be25-1a39f6edd6fa service glance] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:55:23.063774 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513777ba-d13f-4f63-be25-1a39f6edd6fa service glance] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:55:23.064054 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513777ba-d13f-4f63-be25-1a39f6edd6fa service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:55:23.074766 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513777ba-d13f-4f63-be25-1a39f6edd6fa service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:55:23.076698 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513777ba-d13f-4f63-be25-1a39f6edd6fa service glance] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:55:23.088949 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 756/1475] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jun 23 15:55:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 15:55:25.282938 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c539f287-c221-4a0a-a161-04a62bb408ac None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:55:25.283476 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c539f287-c221-4a0a-a161-04a62bb408ac None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:55:25.283476 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c539f287-c221-4a0a-a161-04a62bb408ac None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:55:25.284877 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 720/1476] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jun 23 15:55:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 15:55:25.290565 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d27f22-55f4-4ae0-a699-7115dd20ec6a None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:55:25.290997 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d27f22-55f4-4ae0-a699-7115dd20ec6a None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:55:25.291160 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d27f22-55f4-4ae0-a699-7115dd20ec6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:55:25.308939 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-d8d27f22-55f4-4ae0-a699-7115dd20ec6a None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:55:25.310148 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-d8d27f22-55f4-4ae0-a699-7115dd20ec6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:55:25.318025 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 757/1477] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jun 23 15:55:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 15:55:25.345701 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c539f287-c221-4a0a-a161-04a62bb408ac None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:55:25.357962 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c539f287-c221-4a0a-a161-04a62bb408ac None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:55:25.362279 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a764f42-ec9a-4a5d-b677-636985717ef8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:55:25.362964 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a764f42-ec9a-4a5d-b677-636985717ef8 admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:55:25.363247 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a764f42-ec9a-4a5d-b677-636985717ef8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:55:25.363520 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a764f42-ec9a-4a5d-b677-636985717ef8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:55:25.364354 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a764f42-ec9a-4a5d-b677-636985717ef8 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:55:25.366636 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: WARNING py.warnings [None req-2a764f42-ec9a-4a5d-b677-636985717ef8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 23 15:55:25.366636 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: warnings.warn(msg) Jun 23 15:55:25.366636 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: Jun 23 15:55:25.367792 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a764f42-ec9a-4a5d-b677-636985717ef8 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:55:25.387293 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 721/1478] 10.222.0.46 () {56 vars in 1108 bytes} [Thu Jun 23 15:55:25 2022] GET /identity/v3/projects => generated 2553 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 15:55:27.457436 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c0303a-ec96-46a3-98b2-6d813f313b5e None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:55:27.457977 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c0303a-ec96-46a3-98b2-6d813f313b5e None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:55:27.458444 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c0303a-ec96-46a3-98b2-6d813f313b5e None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:55:27.459996 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 758/1479] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jun 23 15:55:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 15:55:27.465305 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66672c0a-e4fb-43a1-b18c-0004f74603f5 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:55:27.465730 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66672c0a-e4fb-43a1-b18c-0004f74603f5 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:55:27.465870 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66672c0a-e4fb-43a1-b18c-0004f74603f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:55:27.484965 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-66672c0a-e4fb-43a1-b18c-0004f74603f5 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:55:27.486195 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-66672c0a-e4fb-43a1-b18c-0004f74603f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:55:27.494747 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 722/1480] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jun 23 15:55:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 15:55:27.539521 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c0303a-ec96-46a3-98b2-6d813f313b5e None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:55:27.554205 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c0303a-ec96-46a3-98b2-6d813f313b5e None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:55:27.561393 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a5739b-484f-491d-8a70-32c096cd9fa7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '705115d19bab424d894bea8b1b48a7f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:55:27.562048 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a5739b-484f-491d-8a70-32c096cd9fa7 service nova] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:55:27.562329 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a5739b-484f-491d-8a70-32c096cd9fa7 service nova] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:55:27.562604 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a5739b-484f-491d-8a70-32c096cd9fa7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:55:27.571791 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a5739b-484f-491d-8a70-32c096cd9fa7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:55:27.573446 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a5739b-484f-491d-8a70-32c096cd9fa7 service nova] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:55:27.583620 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 759/1481] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jun 23 15:55:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 15:55:29.827546 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60fb25f-7c45-4763-a5f5-7602f55c7408 None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:55:29.828078 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60fb25f-7c45-4763-a5f5-7602f55c7408 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:55:29.828078 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60fb25f-7c45-4763-a5f5-7602f55c7408 None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:55:29.829400 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 723/1482] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jun 23 15:55:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 15:55:29.834577 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4c3583-2cff-47fe-9a58-17f70e493b85 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:55:29.835025 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4c3583-2cff-47fe-9a58-17f70e493b85 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:55:29.835173 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4c3583-2cff-47fe-9a58-17f70e493b85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:55:29.852803 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-ab4c3583-2cff-47fe-9a58-17f70e493b85 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:55:29.854005 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-ab4c3583-2cff-47fe-9a58-17f70e493b85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:55:29.862357 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 760/1483] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jun 23 15:55:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 15:55:29.899990 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60fb25f-7c45-4763-a5f5-7602f55c7408 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:55:29.910621 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60fb25f-7c45-4763-a5f5-7602f55c7408 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:55:29.914212 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3feecaf-f551-40e0-91ae-7fc8d1feef19 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51784064951a4619ae09b9167fb0fd0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:55:29.914889 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3feecaf-f551-40e0-91ae-7fc8d1feef19 service neutron] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:55:29.915178 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3feecaf-f551-40e0-91ae-7fc8d1feef19 service neutron] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:55:29.915461 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3feecaf-f551-40e0-91ae-7fc8d1feef19 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:55:29.924363 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3feecaf-f551-40e0-91ae-7fc8d1feef19 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:55:29.925796 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3feecaf-f551-40e0-91ae-7fc8d1feef19 service neutron] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:55:29.935835 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 724/1484] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jun 23 15:55:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 15:55:32.017338 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69223f2a-69cd-4f02-8938-73aae6683524 None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:55:32.017923 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69223f2a-69cd-4f02-8938-73aae6683524 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:55:32.017923 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69223f2a-69cd-4f02-8938-73aae6683524 None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:55:32.019318 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 761/1485] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jun 23 15:55:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 15:55:32.024864 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4615565b-a9df-4575-8204-4432a1439de6 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:55:32.025296 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4615565b-a9df-4575-8204-4432a1439de6 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:55:32.025432 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4615565b-a9df-4575-8204-4432a1439de6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:55:32.043386 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-4615565b-a9df-4575-8204-4432a1439de6 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:55:32.044593 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-4615565b-a9df-4575-8204-4432a1439de6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:55:32.052325 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 725/1486] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jun 23 15:55:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 15:55:32.090462 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69223f2a-69cd-4f02-8938-73aae6683524 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:55:32.102168 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69223f2a-69cd-4f02-8938-73aae6683524 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:55:32.105801 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae28ec4-c558-4000-b67f-0034ded7ce96 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51784064951a4619ae09b9167fb0fd0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:55:32.106442 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae28ec4-c558-4000-b67f-0034ded7ce96 service neutron] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:55:32.106757 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae28ec4-c558-4000-b67f-0034ded7ce96 service neutron] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:55:32.107036 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae28ec4-c558-4000-b67f-0034ded7ce96 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:55:32.115956 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae28ec4-c558-4000-b67f-0034ded7ce96 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:55:32.117396 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae28ec4-c558-4000-b67f-0034ded7ce96 service neutron] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:55:32.126873 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 762/1487] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jun 23 15:55:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 15:55:34.360007 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d884751f-993b-4fdb-a026-2ca51e3ca0b6 None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:55:34.360583 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d884751f-993b-4fdb-a026-2ca51e3ca0b6 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:55:34.360583 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d884751f-993b-4fdb-a026-2ca51e3ca0b6 None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:55:34.361917 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 726/1488] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jun 23 15:55:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 15:55:34.367836 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa21df05-5ff2-4d80-8ff4-4198b14afbd9 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:55:34.368271 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa21df05-5ff2-4d80-8ff4-4198b14afbd9 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:55:34.368384 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa21df05-5ff2-4d80-8ff4-4198b14afbd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:55:34.386440 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-aa21df05-5ff2-4d80-8ff4-4198b14afbd9 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:55:34.387684 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-aa21df05-5ff2-4d80-8ff4-4198b14afbd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:55:34.395516 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 763/1489] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jun 23 15:55:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 15:55:34.437080 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d884751f-993b-4fdb-a026-2ca51e3ca0b6 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:55:34.449008 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d884751f-993b-4fdb-a026-2ca51e3ca0b6 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:55:34.452658 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db2d064-34f5-4212-82c8-d5b7d997efac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51784064951a4619ae09b9167fb0fd0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:55:34.453299 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db2d064-34f5-4212-82c8-d5b7d997efac service neutron] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:55:34.453586 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db2d064-34f5-4212-82c8-d5b7d997efac service neutron] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:55:34.453864 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db2d064-34f5-4212-82c8-d5b7d997efac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:55:34.462847 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db2d064-34f5-4212-82c8-d5b7d997efac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:55:34.464283 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db2d064-34f5-4212-82c8-d5b7d997efac service neutron] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:55:34.473616 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 727/1490] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jun 23 15:55:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 15:55:36.762078 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d6a196-767f-44e5-8f86-a6553e134b6c None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:55:36.762590 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d6a196-767f-44e5-8f86-a6553e134b6c None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:55:36.762706 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d6a196-767f-44e5-8f86-a6553e134b6c None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:55:36.764070 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 764/1491] 10.222.0.46 () {54 vars in 926 bytes} [Thu Jun 23 15:55:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 15:55:36.769279 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7805600e-6b2f-441c-8d43-a14d9db63f5c None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:55:36.769650 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7805600e-6b2f-441c-8d43-a14d9db63f5c None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:55:36.769838 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7805600e-6b2f-441c-8d43-a14d9db63f5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:55:36.787809 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-7805600e-6b2f-441c-8d43-a14d9db63f5c None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:55:36.789216 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-7805600e-6b2f-441c-8d43-a14d9db63f5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:55:36.797106 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 728/1492] 10.222.0.46 () {58 vars in 1023 bytes} [Thu Jun 23 15:55:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 15:55:36.836887 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d6a196-767f-44e5-8f86-a6553e134b6c None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:55:36.847616 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d6a196-767f-44e5-8f86-a6553e134b6c None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:55:36.851137 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0643327-8f8d-4130-9031-3b9a422c4e8e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51784064951a4619ae09b9167fb0fd0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:55:36.851793 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0643327-8f8d-4130-9031-3b9a422c4e8e service neutron] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:55:36.852077 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0643327-8f8d-4130-9031-3b9a422c4e8e service neutron] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:55:36.852353 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0643327-8f8d-4130-9031-3b9a422c4e8e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:55:36.861527 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0643327-8f8d-4130-9031-3b9a422c4e8e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:55:36.863037 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0643327-8f8d-4130-9031-3b9a422c4e8e service neutron] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:55:36.873594 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 765/1493] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jun 23 15:55:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 15:55:46.782521 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=33 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:55:46.869253 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=33 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:57:04.026415 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff15b30e-b827-46d7-bf61-944aeea2e96a None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:57:04.027044 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff15b30e-b827-46d7-bf61-944aeea2e96a None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:57:04.027044 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff15b30e-b827-46d7-bf61-944aeea2e96a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:57:04.044625 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-ff15b30e-b827-46d7-bf61-944aeea2e96a None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:57:04.045821 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-ff15b30e-b827-46d7-bf61-944aeea2e96a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:57:04.053779 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 729/1494] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 15:57:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 15:57:04.063173 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176bcfa0-1810-41cd-bc56-82376783172c None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:57:04.063684 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176bcfa0-1810-41cd-bc56-82376783172c None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:57:04.063790 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176bcfa0-1810-41cd-bc56-82376783172c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:57:04.081858 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-176bcfa0-1810-41cd-bc56-82376783172c None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:57:04.083092 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-176bcfa0-1810-41cd-bc56-82376783172c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:57:04.091492 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 766/1495] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 15:57:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 15:57:04.096103 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff15b30e-b827-46d7-bf61-944aeea2e96a None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:57:04.105958 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff15b30e-b827-46d7-bf61-944aeea2e96a None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:57:04.110480 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c258bbdf-6c2a-494a-869b-c14fb61c149d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:57:04.111202 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c258bbdf-6c2a-494a-869b-c14fb61c149d admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:57:04.111490 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c258bbdf-6c2a-494a-869b-c14fb61c149d admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:57:04.111768 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c258bbdf-6c2a-494a-869b-c14fb61c149d admin admin] PATH_INFO: `/v3/domains` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:57:04.112514 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c258bbdf-6c2a-494a-869b-c14fb61c149d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 15:57:04.112806 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c258bbdf-6c2a-494a-869b-c14fb61c149d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:57:04.113819 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: WARNING py.warnings [None req-c258bbdf-6c2a-494a-869b-c14fb61c149d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 23 15:57:04.113819 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: warnings.warn(msg) Jun 23 15:57:04.113819 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: Jun 23 15:57:04.114752 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c258bbdf-6c2a-494a-869b-c14fb61c149d admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:57:04.126445 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 730/1496] 10.222.0.46 () {58 vars in 1152 bytes} [Thu Jun 23 15:57:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 15:57:04.135158 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176bcfa0-1810-41cd-bc56-82376783172c None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:57:04.144380 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176bcfa0-1810-41cd-bc56-82376783172c None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:57:04.149917 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995708c7-e3d4-42c1-b51e-1ac3217bed9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:57:04.150572 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995708c7-e3d4-42c1-b51e-1ac3217bed9d admin admin] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:57:04.150888 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995708c7-e3d4-42c1-b51e-1ac3217bed9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:57:04.151169 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995708c7-e3d4-42c1-b51e-1ac3217bed9d admin admin] PATH_INFO: `/v3/projects` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:57:04.151872 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995708c7-e3d4-42c1-b51e-1ac3217bed9d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:57:04.153654 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: WARNING py.warnings [None req-995708c7-e3d4-42c1-b51e-1ac3217bed9d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 23 15:57:04.153654 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: warnings.warn(msg) Jun 23 15:57:04.153654 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: Jun 23 15:57:04.154891 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995708c7-e3d4-42c1-b51e-1ac3217bed9d admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:57:04.198961 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 767/1497] 10.222.0.46 () {60 vars in 1152 bytes} [Thu Jun 23 15:57:04 2022] POST /identity/v3/projects => generated 379 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 15:57:04.203742 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c258bbdf-6c2a-494a-869b-c14fb61c149d admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:57:04.214620 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c258bbdf-6c2a-494a-869b-c14fb61c149d admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:57:04.270418 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3bb84a-490c-48ea-aa70-fda7b76cd7d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:57:04.271135 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3bb84a-490c-48ea-aa70-fda7b76cd7d7 admin admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:57:04.271428 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3bb84a-490c-48ea-aa70-fda7b76cd7d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:57:04.271853 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3bb84a-490c-48ea-aa70-fda7b76cd7d7 admin admin] PATH_INFO: `/v3/users` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:57:04.273091 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3bb84a-490c-48ea-aa70-fda7b76cd7d7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:57:04.274127 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: WARNING py.warnings [None req-1b3bb84a-490c-48ea-aa70-fda7b76cd7d7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 23 15:57:04.274127 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: warnings.warn(msg) Jun 23 15:57:04.274127 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: Jun 23 15:57:04.275724 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3bb84a-490c-48ea-aa70-fda7b76cd7d7 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:57:04.295997 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 731/1498] 10.222.0.46 () {60 vars in 1143 bytes} [Thu Jun 23 15:57:04 2022] POST /identity/v3/users => generated 290 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 15:57:04.300163 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995708c7-e3d4-42c1-b51e-1ac3217bed9d admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:57:04.311286 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995708c7-e3d4-42c1-b51e-1ac3217bed9d admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:57:04.317214 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a8cb50-d725-4409-9cff-67096be5588a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:57:04.318192 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a8cb50-d725-4409-9cff-67096be5588a admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:57:04.318619 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a8cb50-d725-4409-9cff-67096be5588a admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:57:04.319523 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a8cb50-d725-4409-9cff-67096be5588a admin admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:57:04.320397 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a8cb50-d725-4409-9cff-67096be5588a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:57:04.321775 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: WARNING py.warnings [None req-a2a8cb50-d725-4409-9cff-67096be5588a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 23 15:57:04.321775 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: warnings.warn(msg) Jun 23 15:57:04.321775 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: Jun 23 15:57:04.322777 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a8cb50-d725-4409-9cff-67096be5588a admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:57:04.337619 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 768/1499] 10.222.0.46 () {58 vars in 1121 bytes} [Thu Jun 23 15:57:04 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 15:57:04.342006 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3bb84a-490c-48ea-aa70-fda7b76cd7d7 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:57:04.351785 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3bb84a-490c-48ea-aa70-fda7b76cd7d7 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:57:04.356259 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249c8f94-d39a-49ef-9305-eb4f6dc9522d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:57:04.356916 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249c8f94-d39a-49ef-9305-eb4f6dc9522d admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:57:04.357198 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249c8f94-d39a-49ef-9305-eb4f6dc9522d admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:57:04.357472 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249c8f94-d39a-49ef-9305-eb4f6dc9522d admin admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:57:04.358203 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249c8f94-d39a-49ef-9305-eb4f6dc9522d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:57:04.359247 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: WARNING py.warnings [None req-249c8f94-d39a-49ef-9305-eb4f6dc9522d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 23 15:57:04.359247 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: warnings.warn(msg) Jun 23 15:57:04.359247 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: Jun 23 15:57:04.360130 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249c8f94-d39a-49ef-9305-eb4f6dc9522d admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:57:04.373560 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 732/1500] 10.222.0.46 () {58 vars in 1121 bytes} [Thu Jun 23 15:57:04 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 15:57:04.377772 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a8cb50-d725-4409-9cff-67096be5588a admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:57:04.396466 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a8cb50-d725-4409-9cff-67096be5588a admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:57:04.401520 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afedc89-9806-46b9-807f-e161b5129683 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:57:04.402423 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afedc89-9806-46b9-807f-e161b5129683 admin admin] REQUEST_METHOD: `PUT` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:57:04.402732 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afedc89-9806-46b9-807f-e161b5129683 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:57:04.403019 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afedc89-9806-46b9-807f-e161b5129683 admin admin] PATH_INFO: `/v3/projects/e53884fe381d4f5d989188fae0fca078/users/2068b1e5acac4513ae302a2dde81d1eb/roles/e2a5bbf3200a4d4091d72bc7ff78c34d` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:57:04.418369 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afedc89-9806-46b9-807f-e161b5129683 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e53884fe381d4f5d989188fae0fca078, user_id=2068b1e5acac4513ae302a2dde81d1eb, role_id=e2a5bbf3200a4d4091d72bc7ff78c34d)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:57:04.419469 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: WARNING py.warnings [None req-1afedc89-9806-46b9-807f-e161b5129683 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 23 15:57:04.419469 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: warnings.warn(msg) Jun 23 15:57:04.419469 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: Jun 23 15:57:04.421396 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afedc89-9806-46b9-807f-e161b5129683 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:57:04.435520 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 769/1501] 10.222.0.46 () {60 vars in 1482 bytes} [Thu Jun 23 15:57:04 2022] PUT /identity/v3/projects/e53884fe381d4f5d989188fae0fca078/users/2068b1e5acac4513ae302a2dde81d1eb/roles/e2a5bbf3200a4d4091d72bc7ff78c34d => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 15:57:04.447485 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32d4251-ca59-42cd-9e83-b983bb6ce055 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:57:04.447782 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32d4251-ca59-42cd-9e83-b983bb6ce055 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:57:04.448065 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32d4251-ca59-42cd-9e83-b983bb6ce055 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:57:04.473426 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-d32d4251-ca59-42cd-9e83-b983bb6ce055 None None] MFA Rules not processed for user `2068b1e5acac4513ae302a2dde81d1eb`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:57:04.474661 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-d32d4251-ca59-42cd-9e83-b983bb6ce055 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:57:04.543524 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 733/1502] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 15:57:04 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 15:57:04.557093 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a452abd1-e7b2-47d3-af85-0e4f7c68f36f None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:57:04.557489 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a452abd1-e7b2-47d3-af85-0e4f7c68f36f None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:57:04.557807 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a452abd1-e7b2-47d3-af85-0e4f7c68f36f None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:57:04.559227 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 770/1503] 10.222.0.46 () {54 vars in 963 bytes} [Thu Jun 23 15:57:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 15:57:04.564461 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c848e4e-3c91-465d-9b8a-1ea4ecd1785c None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:57:04.564760 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c848e4e-3c91-465d-9b8a-1ea4ecd1785c None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:57:04.565045 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c848e4e-3c91-465d-9b8a-1ea4ecd1785c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:57:04.583728 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-4c848e4e-3c91-465d-9b8a-1ea4ecd1785c None None] MFA Rules not processed for user `ce18eaaba3a6477684fff1fd98ec8b51`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:57:04.584941 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-4c848e4e-3c91-465d-9b8a-1ea4ecd1785c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:57:04.627523 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:57:04.636878 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 734/1504] 10.222.0.46 () {58 vars in 1060 bytes} [Thu Jun 23 15:57:04 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 15:57:04.644665 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a452abd1-e7b2-47d3-af85-0e4f7c68f36f None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:57:04.655521 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a452abd1-e7b2-47d3-af85-0e4f7c68f36f None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:57:04.659002 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cbec9e-e10d-482d-b3e4-650d00d3d428 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce18eaaba3a6477684fff1fd98ec8b51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:57:04.659629 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cbec9e-e10d-482d-b3e4-650d00d3d428 service cinder] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:57:04.659910 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cbec9e-e10d-482d-b3e4-650d00d3d428 service cinder] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:57:04.660182 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cbec9e-e10d-482d-b3e4-650d00d3d428 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:57:04.668545 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46cbec9e-e10d-482d-b3e4-650d00d3d428 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:57:04.670018 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46cbec9e-e10d-482d-b3e4-650d00d3d428 service cinder] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:57:04.679511 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 771/1505] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jun 23 15:57:04 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 15:57:04.721389 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2960659d-295c-4f33-9778-244934ac044d None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:57:04.721682 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2960659d-295c-4f33-9778-244934ac044d None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:57:04.721969 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2960659d-295c-4f33-9778-244934ac044d None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:57:04.723297 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 735/1506] 10.222.0.46 () {52 vars in 829 bytes} [Thu Jun 23 15:57:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 15:57:04.727527 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cbec9e-e10d-482d-b3e4-650d00d3d428 service cinder] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:57:04.789593 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cbec9e-e10d-482d-b3e4-650d00d3d428 service cinder] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:57:04.794160 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ec943d-9a18-4186-8b49-d3101df7ed00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:57:04.795079 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ec943d-9a18-4186-8b49-d3101df7ed00 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:57:04.795362 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ec943d-9a18-4186-8b49-d3101df7ed00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:57:04.795644 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ec943d-9a18-4186-8b49-d3101df7ed00 admin admin] PATH_INFO: `/v3/users/2068b1e5acac4513ae302a2dde81d1eb` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:57:04.796881 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ec943d-9a18-4186-8b49-d3101df7ed00 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2068b1e5acac4513ae302a2dde81d1eb)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:57:04.798014 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: WARNING py.warnings [None req-b8ec943d-9a18-4186-8b49-d3101df7ed00 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 23 15:57:04.798014 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: warnings.warn(msg) Jun 23 15:57:04.798014 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: Jun 23 15:57:04.799273 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ec943d-9a18-4186-8b49-d3101df7ed00 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:57:04.866709 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-b8ec943d-9a18-4186-8b49-d3101df7ed00 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2068b1e5acac4513ae302a2dde81d1eb'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 15:57:04.879667 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-b8ec943d-9a18-4186-8b49-d3101df7ed00 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2068b1e5acac4513ae302a2dde81d1eb'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 15:57:04.880477 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-b8ec943d-9a18-4186-8b49-d3101df7ed00 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2068b1e5acac4513ae302a2dde81d1eb'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 15:57:04.881100 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-b8ec943d-9a18-4186-8b49-d3101df7ed00 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2068b1e5acac4513ae302a2dde81d1eb'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 15:57:04.903228 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-b8ec943d-9a18-4186-8b49-d3101df7ed00 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2068b1e5acac4513ae302a2dde81d1eb'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 15:57:04.907269 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:57:04.921919 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 772/1507] 10.222.0.46 () {58 vars in 1223 bytes} [Thu Jun 23 15:57:04 2022] DELETE /identity/v3/users/2068b1e5acac4513ae302a2dde81d1eb => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 15:57:04.926373 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3942b0d-b1b8-4da8-9086-838182caba7e None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:57:04.926779 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3942b0d-b1b8-4da8-9086-838182caba7e None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:57:04.927071 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3942b0d-b1b8-4da8-9086-838182caba7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:57:04.945189 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-f3942b0d-b1b8-4da8-9086-838182caba7e None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:57:04.946412 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-f3942b0d-b1b8-4da8-9086-838182caba7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:57:05.004694 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 736/1508] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 15:57:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 15:57:05.014165 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ec943d-9a18-4186-8b49-d3101df7ed00 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:57:05.015616 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-b8ec943d-9a18-4186-8b49-d3101df7ed00 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:57:05.075274 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ec943d-9a18-4186-8b49-d3101df7ed00 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:57:05.079005 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbf7ae3-7152-4be8-8bca-967ff9ba575f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51784064951a4619ae09b9167fb0fd0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:57:05.079564 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbf7ae3-7152-4be8-8bca-967ff9ba575f service neutron] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:57:05.079845 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbf7ae3-7152-4be8-8bca-967ff9ba575f service neutron] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:57:05.080119 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbf7ae3-7152-4be8-8bca-967ff9ba575f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:57:05.089743 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbf7ae3-7152-4be8-8bca-967ff9ba575f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:57:05.091230 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbf7ae3-7152-4be8-8bca-967ff9ba575f service neutron] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:57:05.101122 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 773/1509] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jun 23 15:57:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 15:57:05.458011 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3942b0d-b1b8-4da8-9086-838182caba7e None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:57:05.460135 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-f3942b0d-b1b8-4da8-9086-838182caba7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:57:05.472854 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3942b0d-b1b8-4da8-9086-838182caba7e None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:57:05.478017 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42286650-5e96-4f8a-b1c2-efd52e246d04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:57:05.479006 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42286650-5e96-4f8a-b1c2-efd52e246d04 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:57:05.479373 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42286650-5e96-4f8a-b1c2-efd52e246d04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:57:05.479716 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42286650-5e96-4f8a-b1c2-efd52e246d04 admin admin] PATH_INFO: `/v3/projects/e53884fe381d4f5d989188fae0fca078` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:57:05.481125 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42286650-5e96-4f8a-b1c2-efd52e246d04 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e53884fe381d4f5d989188fae0fca078)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:57:05.482249 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: WARNING py.warnings [None req-42286650-5e96-4f8a-b1c2-efd52e246d04 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 23 15:57:05.482249 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: warnings.warn(msg) Jun 23 15:57:05.482249 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: Jun 23 15:57:05.483734 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42286650-5e96-4f8a-b1c2-efd52e246d04 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:57:05.557383 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-42286650-5e96-4f8a-b1c2-efd52e246d04 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e53884fe381d4f5d989188fae0fca078', 'request_id': 'req-42286650-5e96-4f8a-b1c2-efd52e246d04'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 15:57:05.564300 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-42286650-5e96-4f8a-b1c2-efd52e246d04 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e53884fe381d4f5d989188fae0fca078', 'request_id': 'req-42286650-5e96-4f8a-b1c2-efd52e246d04'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 15:57:05.574476 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-42286650-5e96-4f8a-b1c2-efd52e246d04 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e53884fe381d4f5d989188fae0fca078', 'request_id': 'req-42286650-5e96-4f8a-b1c2-efd52e246d04'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 15:57:05.593645 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-42286650-5e96-4f8a-b1c2-efd52e246d04 admin admin] The token cache is being invalidate because project e53884fe381d4f5d989188fae0fca078 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80154) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 15:57:05.593867 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-42286650-5e96-4f8a-b1c2-efd52e246d04 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 15:57:05.594522 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-42286650-5e96-4f8a-b1c2-efd52e246d04 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 15:57:05.596216 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 737/1510] 10.222.0.46 () {58 vars in 1232 bytes} [Thu Jun 23 15:57:05 2022] DELETE /identity/v3/projects/e53884fe381d4f5d989188fae0fca078 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 15:57:15.096408 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:DELETE=1,keystone:SELECT=28 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:57:15.590486 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=55,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:58:43.983289 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a196a12-8b98-4d9c-a2b1-c1d581c5575b None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:58:43.983918 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a196a12-8b98-4d9c-a2b1-c1d581c5575b None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:58:43.983918 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a196a12-8b98-4d9c-a2b1-c1d581c5575b None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:58:43.986735 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 774/1511] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 23 15:58:43 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 15:58:43.994586 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1037929-979c-4879-b005-626a1a71b5fc None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:58:43.995192 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1037929-979c-4879-b005-626a1a71b5fc None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:58:43.995192 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1037929-979c-4879-b005-626a1a71b5fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:58:44.013208 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-f1037929-979c-4879-b005-626a1a71b5fc None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:58:44.014433 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-f1037929-979c-4879-b005-626a1a71b5fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:58:44.075498 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 738/1512] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 23 15:58:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 15:58:44.233153 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a196a12-8b98-4d9c-a2b1-c1d581c5575b None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:58:44.234622 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-9a196a12-8b98-4d9c-a2b1-c1d581c5575b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:58:44.308523 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a196a12-8b98-4d9c-a2b1-c1d581c5575b None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:58:44.313926 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd662045-8748-45f9-ab27-fb4bf42cd556 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '705115d19bab424d894bea8b1b48a7f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:58:44.314576 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd662045-8748-45f9-ab27-fb4bf42cd556 service nova] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:58:44.314885 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd662045-8748-45f9-ab27-fb4bf42cd556 service nova] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:58:44.315163 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd662045-8748-45f9-ab27-fb4bf42cd556 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:58:44.323424 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd662045-8748-45f9-ab27-fb4bf42cd556 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:58:44.325032 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd662045-8748-45f9-ab27-fb4bf42cd556 service nova] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:58:44.334419 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 775/1513] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jun 23 15:58:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 15:58:45.202536 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56339af8-d14a-4c14-bb48-5a078e0c9a4a None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:58:45.203124 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56339af8-d14a-4c14-bb48-5a078e0c9a4a None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:58:45.203124 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56339af8-d14a-4c14-bb48-5a078e0c9a4a None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:58:45.204408 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 739/1514] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 23 15:58:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 15:58:45.209419 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1f92e5-9ef7-418c-9002-d6753b1f5bba None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:58:45.209879 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1f92e5-9ef7-418c-9002-d6753b1f5bba None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:58:45.209997 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1f92e5-9ef7-418c-9002-d6753b1f5bba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:58:45.228925 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-1f1f92e5-9ef7-418c-9002-d6753b1f5bba None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:58:45.230174 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-1f1f92e5-9ef7-418c-9002-d6753b1f5bba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:58:45.239443 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 776/1515] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 23 15:58:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 15:58:45.403156 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56339af8-d14a-4c14-bb48-5a078e0c9a4a None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:58:45.418342 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56339af8-d14a-4c14-bb48-5a078e0c9a4a None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:58:45.423985 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac026666-9659-4747-ba76-1a0f05e507a2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '705115d19bab424d894bea8b1b48a7f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:58:45.424639 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac026666-9659-4747-ba76-1a0f05e507a2 service nova] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:58:45.424927 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac026666-9659-4747-ba76-1a0f05e507a2 service nova] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:58:45.425211 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac026666-9659-4747-ba76-1a0f05e507a2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:58:45.434671 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac026666-9659-4747-ba76-1a0f05e507a2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:58:45.436460 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac026666-9659-4747-ba76-1a0f05e507a2 service nova] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:58:45.446089 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 740/1516] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jun 23 15:58:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 15:58:47.722591 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3efc64-fb02-428b-bcf2-6cebfc413d52 None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:58:47.723201 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3efc64-fb02-428b-bcf2-6cebfc413d52 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:58:47.723360 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3efc64-fb02-428b-bcf2-6cebfc413d52 None None] PATH_INFO: `/v3` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:58:47.725359 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 777/1517] 10.222.0.44 () {54 vars in 925 bytes} [Thu Jun 23 15:58:47 2022] GET /identity/v3 => generated 255 bytes in 4 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 15:58:47.732526 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ded34f-27ab-48c2-922a-bac3be0c1d97 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:58:47.733015 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ded34f-27ab-48c2-922a-bac3be0c1d97 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:58:47.733116 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ded34f-27ab-48c2-922a-bac3be0c1d97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:58:47.752557 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-52ded34f-27ab-48c2-922a-bac3be0c1d97 None None] MFA Rules not processed for user `3f66988b73644fe38b9849663ee70f25`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:58:47.753799 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-52ded34f-27ab-48c2-922a-bac3be0c1d97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:58:47.817040 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 741/1518] 10.222.0.44 () {58 vars in 1015 bytes} [Thu Jun 23 15:58:47 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 15:58:51.917505 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a4d10a-f48b-44ee-91a4-053971c5179c None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:58:51.918068 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a4d10a-f48b-44ee-91a4-053971c5179c None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:58:51.918454 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a4d10a-f48b-44ee-91a4-053971c5179c None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:58:51.920967 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 778/1519] 10.222.0.46 () {54 vars in 960 bytes} [Thu Jun 23 15:58:51 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 15:58:51.927944 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899caba8-73a9-491e-9c1f-17aac0c96b5b None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:58:51.928486 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899caba8-73a9-491e-9c1f-17aac0c96b5b None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:58:51.928699 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899caba8-73a9-491e-9c1f-17aac0c96b5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:58:51.948648 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-899caba8-73a9-491e-9c1f-17aac0c96b5b None None] MFA Rules not processed for user `3f66988b73644fe38b9849663ee70f25`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:58:51.949909 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-899caba8-73a9-491e-9c1f-17aac0c96b5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:58:51.958667 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 742/1520] 10.222.0.46 () {58 vars in 1057 bytes} [Thu Jun 23 15:58:51 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 15:58:51.968578 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a4d10a-f48b-44ee-91a4-053971c5179c None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:58:51.983905 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a4d10a-f48b-44ee-91a4-053971c5179c None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:58:51.989072 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d217eb-5da7-4740-9406-d138ed9a2b77 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f66988b73644fe38b9849663ee70f25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:58:51.989737 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d217eb-5da7-4740-9406-d138ed9a2b77 service placement] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:58:51.990031 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d217eb-5da7-4740-9406-d138ed9a2b77 service placement] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:58:51.990291 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d217eb-5da7-4740-9406-d138ed9a2b77 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:58:52.000916 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d217eb-5da7-4740-9406-d138ed9a2b77 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:58:52.002539 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d217eb-5da7-4740-9406-d138ed9a2b77 service placement] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:58:52.014194 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 779/1521] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jun 23 15:58:51 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 15:58:56.752384 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d97d855-c5c6-4d07-8a19-0334ad3931f1 None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:58:56.752945 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d97d855-c5c6-4d07-8a19-0334ad3931f1 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:58:56.752945 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d97d855-c5c6-4d07-8a19-0334ad3931f1 None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:58:56.754234 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 743/1522] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 23 15:58:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 15:58:56.760570 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96777703-c0b2-45b0-b7a8-d3c475d34ea1 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:58:56.761234 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96777703-c0b2-45b0-b7a8-d3c475d34ea1 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:58:56.761393 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96777703-c0b2-45b0-b7a8-d3c475d34ea1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:58:56.785084 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-96777703-c0b2-45b0-b7a8-d3c475d34ea1 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:58:56.786324 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-96777703-c0b2-45b0-b7a8-d3c475d34ea1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:58:56.795260 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 780/1523] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 23 15:58:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 15:58:56.968259 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d97d855-c5c6-4d07-8a19-0334ad3931f1 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:58:56.993453 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d97d855-c5c6-4d07-8a19-0334ad3931f1 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:58:57.002825 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee592c02-230c-4cbf-90a5-dced9a5e7194 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '705115d19bab424d894bea8b1b48a7f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:58:57.003879 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee592c02-230c-4cbf-90a5-dced9a5e7194 service nova] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:58:57.004376 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee592c02-230c-4cbf-90a5-dced9a5e7194 service nova] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:58:57.004872 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee592c02-230c-4cbf-90a5-dced9a5e7194 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:58:57.021540 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee592c02-230c-4cbf-90a5-dced9a5e7194 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:58:57.024759 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee592c02-230c-4cbf-90a5-dced9a5e7194 service nova] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:58:57.042251 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 744/1524] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jun 23 15:58:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 74 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 15:58:57.950605 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2a5734-0d53-4763-a9a6-c1ec38e3d846 None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:58:57.951249 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2a5734-0d53-4763-a9a6-c1ec38e3d846 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:58:57.951249 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2a5734-0d53-4763-a9a6-c1ec38e3d846 None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:58:57.952662 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 781/1525] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 23 15:58:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 15:58:57.959659 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fffa6f-95b2-45bb-b053-18c273752ccd None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:58:57.960447 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fffa6f-95b2-45bb-b053-18c273752ccd None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:58:57.960447 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fffa6f-95b2-45bb-b053-18c273752ccd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:58:57.980764 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-55fffa6f-95b2-45bb-b053-18c273752ccd None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:58:57.982009 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-55fffa6f-95b2-45bb-b053-18c273752ccd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:58:57.992457 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 745/1526] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 23 15:58:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 15:58:58.239082 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2a5734-0d53-4763-a9a6-c1ec38e3d846 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:58:58.261387 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2a5734-0d53-4763-a9a6-c1ec38e3d846 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:58:58.269902 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d61cac-1fbb-4149-8609-468347e7906d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '705115d19bab424d894bea8b1b48a7f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:58:58.270992 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d61cac-1fbb-4149-8609-468347e7906d service nova] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:58:58.271458 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d61cac-1fbb-4149-8609-468347e7906d service nova] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:58:58.271927 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d61cac-1fbb-4149-8609-468347e7906d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:58:58.282254 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d61cac-1fbb-4149-8609-468347e7906d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:58:58.284043 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d61cac-1fbb-4149-8609-468347e7906d service nova] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:58:58.298925 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 782/1527] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jun 23 15:58:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 15:59:02.233718 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abae1a9-0760-4de8-a4e3-82b264066a83 None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:59:02.234336 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abae1a9-0760-4de8-a4e3-82b264066a83 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:59:02.234336 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abae1a9-0760-4de8-a4e3-82b264066a83 None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:59:02.235704 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 746/1528] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 23 15:59:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 15:59:02.241790 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e266ccd0-addd-4b16-9f1d-34c26f66c927 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:59:02.242336 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e266ccd0-addd-4b16-9f1d-34c26f66c927 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:59:02.242468 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e266ccd0-addd-4b16-9f1d-34c26f66c927 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:59:02.261116 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-e266ccd0-addd-4b16-9f1d-34c26f66c927 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:59:02.262327 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-e266ccd0-addd-4b16-9f1d-34c26f66c927 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:59:02.270707 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 783/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 23 15:59:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 15:59:02.315864 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abae1a9-0760-4de8-a4e3-82b264066a83 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:59:02.317609 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-0abae1a9-0760-4de8-a4e3-82b264066a83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:59:02.378768 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abae1a9-0760-4de8-a4e3-82b264066a83 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:59:02.383089 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b069b26-4127-42c3-85c9-e43a59ab0c63 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51784064951a4619ae09b9167fb0fd0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:59:02.383767 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b069b26-4127-42c3-85c9-e43a59ab0c63 service neutron] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:59:02.384060 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b069b26-4127-42c3-85c9-e43a59ab0c63 service neutron] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:59:02.384346 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b069b26-4127-42c3-85c9-e43a59ab0c63 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:59:02.394245 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b069b26-4127-42c3-85c9-e43a59ab0c63 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:59:02.395729 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b069b26-4127-42c3-85c9-e43a59ab0c63 service neutron] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:59:02.401895 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:59:02.407342 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 747/1530] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jun 23 15:59:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 15:59:04.489248 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002b1799-b5be-475e-bdca-e3c6ee96c099 None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:59:04.489825 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002b1799-b5be-475e-bdca-e3c6ee96c099 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:59:04.489825 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002b1799-b5be-475e-bdca-e3c6ee96c099 None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:59:04.491228 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 784/1531] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 23 15:59:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 15:59:04.498323 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6e5aba-39a2-4646-a975-bfcdc359aa2d None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:59:04.498834 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6e5aba-39a2-4646-a975-bfcdc359aa2d None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:59:04.498949 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6e5aba-39a2-4646-a975-bfcdc359aa2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:59:04.523820 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-6e6e5aba-39a2-4646-a975-bfcdc359aa2d None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:59:04.525044 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-6e6e5aba-39a2-4646-a975-bfcdc359aa2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:59:04.533574 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 748/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 23 15:59:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 15:59:04.572836 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002b1799-b5be-475e-bdca-e3c6ee96c099 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:59:04.574643 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-002b1799-b5be-475e-bdca-e3c6ee96c099 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:59:04.588066 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002b1799-b5be-475e-bdca-e3c6ee96c099 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:59:04.594894 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2d8477-a0de-4eb0-8d7a-45c780ebde00 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51784064951a4619ae09b9167fb0fd0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:59:04.594894 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2d8477-a0de-4eb0-8d7a-45c780ebde00 service neutron] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:59:04.594894 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2d8477-a0de-4eb0-8d7a-45c780ebde00 service neutron] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:59:04.594894 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2d8477-a0de-4eb0-8d7a-45c780ebde00 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:59:04.602998 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2d8477-a0de-4eb0-8d7a-45c780ebde00 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:59:04.604418 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2d8477-a0de-4eb0-8d7a-45c780ebde00 service neutron] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:59:04.614481 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 785/1533] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jun 23 15:59:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 15:59:08.154862 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13639aaa-723c-4014-aedd-e93d0d96ffb9 None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:59:08.155413 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13639aaa-723c-4014-aedd-e93d0d96ffb9 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:59:08.155413 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13639aaa-723c-4014-aedd-e93d0d96ffb9 None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:59:08.156777 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 749/1534] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 23 15:59:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 15:59:08.162243 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f3528d-3f13-451a-b022-09e2535eb0df None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:59:08.162709 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f3528d-3f13-451a-b022-09e2535eb0df None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:59:08.162904 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f3528d-3f13-451a-b022-09e2535eb0df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:59:08.169848 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=62 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:59:08.182212 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-15f3528d-3f13-451a-b022-09e2535eb0df None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:59:08.183446 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-15f3528d-3f13-451a-b022-09e2535eb0df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:59:08.191363 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 786/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 23 15:59:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 15:59:08.495461 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13639aaa-723c-4014-aedd-e93d0d96ffb9 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:59:08.497206 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-13639aaa-723c-4014-aedd-e93d0d96ffb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:59:08.599322 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13639aaa-723c-4014-aedd-e93d0d96ffb9 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:59:08.605353 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5addbe-b49d-4d14-b7a5-36128fc65f2d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce18eaaba3a6477684fff1fd98ec8b51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:59:08.606126 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5addbe-b49d-4d14-b7a5-36128fc65f2d service cinder] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:59:08.606493 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5addbe-b49d-4d14-b7a5-36128fc65f2d service cinder] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:59:08.606811 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5addbe-b49d-4d14-b7a5-36128fc65f2d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:59:08.616782 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5addbe-b49d-4d14-b7a5-36128fc65f2d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:59:08.618241 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5addbe-b49d-4d14-b7a5-36128fc65f2d service cinder] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:59:08.628117 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 750/1536] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jun 23 15:59:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 15:59:10.781574 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e28b90-e3b6-4c27-b343-4e36e2a5099a None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:59:10.782143 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e28b90-e3b6-4c27-b343-4e36e2a5099a None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:59:10.782143 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e28b90-e3b6-4c27-b343-4e36e2a5099a None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:59:10.783568 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 787/1537] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 23 15:59:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 15:59:10.789717 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c837fcd3-87b3-438b-8825-c0de862cba3c None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:59:10.790345 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c837fcd3-87b3-438b-8825-c0de862cba3c None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:59:10.790345 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c837fcd3-87b3-438b-8825-c0de862cba3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:59:10.811178 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-c837fcd3-87b3-438b-8825-c0de862cba3c None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 15:59:10.812416 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-c837fcd3-87b3-438b-8825-c0de862cba3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:59:10.820436 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 751/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 23 15:59:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 15:59:11.132809 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e28b90-e3b6-4c27-b343-4e36e2a5099a None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 15:59:11.134579 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-17e28b90-e3b6-4c27-b343-4e36e2a5099a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 15:59:11.148494 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e28b90-e3b6-4c27-b343-4e36e2a5099a None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 15:59:11.152351 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca427a5-d667-4122-9e7d-33d48afb59de service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce18eaaba3a6477684fff1fd98ec8b51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 15:59:11.153001 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca427a5-d667-4122-9e7d-33d48afb59de service cinder] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 15:59:11.153287 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca427a5-d667-4122-9e7d-33d48afb59de service cinder] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 15:59:11.153573 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca427a5-d667-4122-9e7d-33d48afb59de service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 15:59:11.163072 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca427a5-d667-4122-9e7d-33d48afb59de service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 15:59:11.165140 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca427a5-d667-4122-9e7d-33d48afb59de service cinder] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 15:59:11.174510 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 788/1539] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jun 23 15:59:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 15:59:20.803358 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=32 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 15:59:21.170446 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=8 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:00:40.867447 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e934efe8-e9c2-4746-a595-9de125f4aa29 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:00:40.868978 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e934efe8-e9c2-4746-a595-9de125f4aa29 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:00:40.868978 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e934efe8-e9c2-4746-a595-9de125f4aa29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:00:40.901606 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-e934efe8-e9c2-4746-a595-9de125f4aa29 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:00:40.902929 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-e934efe8-e9c2-4746-a595-9de125f4aa29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:00:40.913515 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 752/1540] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:00:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) Jun 23 16:00:40.927556 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1b0438-afa4-4ca0-9c00-cec20603a5b7 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:00:40.928458 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1b0438-afa4-4ca0-9c00-cec20603a5b7 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:00:40.928458 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1b0438-afa4-4ca0-9c00-cec20603a5b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:00:40.960883 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-fd1b0438-afa4-4ca0-9c00-cec20603a5b7 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:00:40.962419 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-fd1b0438-afa4-4ca0-9c00-cec20603a5b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:00:40.976154 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 789/1541] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:00:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 16:00:40.981446 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e934efe8-e9c2-4746-a595-9de125f4aa29 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:00:40.998010 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e934efe8-e9c2-4746-a595-9de125f4aa29 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:00:41.006204 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775afa75-37c9-4822-ba14-abe22511a0a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:00:41.006942 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775afa75-37c9-4822-ba14-abe22511a0a3 admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:00:41.007269 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775afa75-37c9-4822-ba14-abe22511a0a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:00:41.007980 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775afa75-37c9-4822-ba14-abe22511a0a3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:00:41.008747 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775afa75-37c9-4822-ba14-abe22511a0a3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 16:00:41.009172 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f9939f-4800-4673-b81c-f5ba5023e268 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:00:41.009357 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775afa75-37c9-4822-ba14-abe22511a0a3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:00:41.009525 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f9939f-4800-4673-b81c-f5ba5023e268 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:00:41.009816 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f9939f-4800-4673-b81c-f5ba5023e268 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:00:41.011499 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775afa75-37c9-4822-ba14-abe22511a0a3 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:00:41.030683 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 753/1542] 10.222.0.46 () {58 vars in 1152 bytes} [Thu Jun 23 16:00:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 16:00:41.039298 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775afa75-37c9-4822-ba14-abe22511a0a3 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:00:41.041919 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-d5f9939f-4800-4673-b81c-f5ba5023e268 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:00:41.043479 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-d5f9939f-4800-4673-b81c-f5ba5023e268 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:00:41.053031 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775afa75-37c9-4822-ba14-abe22511a0a3 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:00:41.059757 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 790/1543] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:00:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) Jun 23 16:00:41.061364 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3eeaaf-85c4-4c76-87a5-b3dd4677e033 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:00:41.061658 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3eeaaf-85c4-4c76-87a5-b3dd4677e033 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:00:41.061928 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3eeaaf-85c4-4c76-87a5-b3dd4677e033 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:00:41.066121 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf762d7-a50a-42d1-aa9a-0c88ba7cd7b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:00:41.066864 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf762d7-a50a-42d1-aa9a-0c88ba7cd7b1 admin admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:00:41.067170 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf762d7-a50a-42d1-aa9a-0c88ba7cd7b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:00:41.067455 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf762d7-a50a-42d1-aa9a-0c88ba7cd7b1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:00:41.068246 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf762d7-a50a-42d1-aa9a-0c88ba7cd7b1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:00:41.069290 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: WARNING py.warnings [None req-bcf762d7-a50a-42d1-aa9a-0c88ba7cd7b1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 23 16:00:41.069290 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: warnings.warn(msg) Jun 23 16:00:41.069290 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: Jun 23 16:00:41.070497 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf762d7-a50a-42d1-aa9a-0c88ba7cd7b1 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:00:41.092991 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-5c3eeaaf-85c4-4c76-87a5-b3dd4677e033 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:00:41.094322 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-5c3eeaaf-85c4-4c76-87a5-b3dd4677e033 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:00:41.110331 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 791/1544] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:00:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 16:00:41.111789 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd33f152-67f9-40a3-90aa-260f411b4ae3 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:00:41.112133 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd33f152-67f9-40a3-90aa-260f411b4ae3 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:00:41.112322 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd33f152-67f9-40a3-90aa-260f411b4ae3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:00:41.149106 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-bd33f152-67f9-40a3-90aa-260f411b4ae3 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:00:41.150070 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 754/1545] 10.222.0.46 () {60 vars in 1152 bytes} [Thu Jun 23 16:00:41 2022] POST /identity/v3/projects => generated 377 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 16:00:41.151850 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e132f30-0ff3-4325-b2bb-1efcfbfc4fb1 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:00:41.152210 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e132f30-0ff3-4325-b2bb-1efcfbfc4fb1 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:00:41.152502 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e132f30-0ff3-4325-b2bb-1efcfbfc4fb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:00:41.155223 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-bd33f152-67f9-40a3-90aa-260f411b4ae3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:00:41.199791 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-4e132f30-0ff3-4325-b2bb-1efcfbfc4fb1 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:00:41.201037 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-4e132f30-0ff3-4325-b2bb-1efcfbfc4fb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:00:41.286318 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 792/1546] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:00:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 16:00:41.289378 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01529f67-6afb-401a-8299-a4556b5a2c84 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:00:41.290100 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01529f67-6afb-401a-8299-a4556b5a2c84 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:00:41.290447 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01529f67-6afb-401a-8299-a4556b5a2c84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:00:41.333828 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-01529f67-6afb-401a-8299-a4556b5a2c84 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:00:41.336922 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-01529f67-6afb-401a-8299-a4556b5a2c84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:00:41.357953 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 793/1547] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:00:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 16:00:41.360526 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01529f67-6afb-401a-8299-a4556b5a2c84 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:00:41.382652 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01529f67-6afb-401a-8299-a4556b5a2c84 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:00:41.392086 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa97f1d5-6bdf-42af-8ef5-b317cea11834 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:00:41.393202 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 755/1548] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:00:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 16:00:41.394308 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319a40cb-c5f5-443f-83de-e45c88dd2c2b None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:00:41.394595 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319a40cb-c5f5-443f-83de-e45c88dd2c2b None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:00:41.394906 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319a40cb-c5f5-443f-83de-e45c88dd2c2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:00:41.397404 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa97f1d5-6bdf-42af-8ef5-b317cea11834 admin admin] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:00:41.397983 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa97f1d5-6bdf-42af-8ef5-b317cea11834 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:00:41.398485 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa97f1d5-6bdf-42af-8ef5-b317cea11834 admin admin] PATH_INFO: `/v3/users` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:00:41.402459 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa97f1d5-6bdf-42af-8ef5-b317cea11834 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:00:41.404356 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: WARNING py.warnings [None req-aa97f1d5-6bdf-42af-8ef5-b317cea11834 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 23 16:00:41.404356 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: warnings.warn(msg) Jun 23 16:00:41.404356 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: Jun 23 16:00:41.407493 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa97f1d5-6bdf-42af-8ef5-b317cea11834 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:00:41.445915 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-319a40cb-c5f5-443f-83de-e45c88dd2c2b None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:00:41.448870 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-319a40cb-c5f5-443f-83de-e45c88dd2c2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:00:41.460512 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 794/1549] 10.222.0.46 () {60 vars in 1143 bytes} [Thu Jun 23 16:00:41 2022] POST /identity/v3/users => generated 289 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 16:00:41.461380 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa97f1d5-6bdf-42af-8ef5-b317cea11834 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:00:41.470869 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 756/1550] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:00:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 16:00:41.471957 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319a40cb-c5f5-443f-83de-e45c88dd2c2b None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:00:41.480820 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa97f1d5-6bdf-42af-8ef5-b317cea11834 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:00:41.486064 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a03119-d3fb-41b9-988c-14f9c287875a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:00:41.486905 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a03119-d3fb-41b9-988c-14f9c287875a admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:00:41.487185 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a03119-d3fb-41b9-988c-14f9c287875a admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:00:41.487464 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a03119-d3fb-41b9-988c-14f9c287875a admin admin] PATH_INFO: `/v3/domains` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:00:41.488198 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a03119-d3fb-41b9-988c-14f9c287875a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 16:00:41.488501 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a03119-d3fb-41b9-988c-14f9c287875a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:00:41.489534 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: WARNING py.warnings [None req-c4a03119-d3fb-41b9-988c-14f9c287875a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 23 16:00:41.489534 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: warnings.warn(msg) Jun 23 16:00:41.489534 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: Jun 23 16:00:41.490442 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a03119-d3fb-41b9-988c-14f9c287875a admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:00:41.500275 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319a40cb-c5f5-443f-83de-e45c88dd2c2b None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:00:41.503927 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 795/1551] 10.222.0.46 () {58 vars in 1152 bytes} [Thu Jun 23 16:00:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 16:00:41.505332 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19cede2-dcfa-4c64-8b64-d87ff5d604c1 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:00:41.505613 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19cede2-dcfa-4c64-8b64-d87ff5d604c1 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:00:41.505889 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19cede2-dcfa-4c64-8b64-d87ff5d604c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:00:41.507299 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf22ab5-59be-4d51-ab8c-6c336638377d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:00:41.508323 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf22ab5-59be-4d51-ab8c-6c336638377d admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:00:41.508827 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf22ab5-59be-4d51-ab8c-6c336638377d admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:00:41.509325 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf22ab5-59be-4d51-ab8c-6c336638377d admin admin] PATH_INFO: `/v3/domains` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:00:41.510799 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf22ab5-59be-4d51-ab8c-6c336638377d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 16:00:41.511412 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf22ab5-59be-4d51-ab8c-6c336638377d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:00:41.514846 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf22ab5-59be-4d51-ab8c-6c336638377d admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:00:41.529163 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-f19cede2-dcfa-4c64-8b64-d87ff5d604c1 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:00:41.530633 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-f19cede2-dcfa-4c64-8b64-d87ff5d604c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:00:41.538215 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 757/1552] 10.222.0.46 () {58 vars in 1152 bytes} [Thu Jun 23 16:00:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 16:00:41.540644 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114aa7b9-3fef-499e-9cc8-0d4f493daf0b None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:00:41.541167 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114aa7b9-3fef-499e-9cc8-0d4f493daf0b None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:00:41.542192 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114aa7b9-3fef-499e-9cc8-0d4f493daf0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:00:41.543268 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 796/1553] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:00:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 16:00:41.553037 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd7bf31-ee26-4900-9aa2-16ed35c9d0d7 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:00:41.553676 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd7bf31-ee26-4900-9aa2-16ed35c9d0d7 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:00:41.554377 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd7bf31-ee26-4900-9aa2-16ed35c9d0d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:00:41.570250 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-114aa7b9-3fef-499e-9cc8-0d4f493daf0b None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:00:41.571611 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-114aa7b9-3fef-499e-9cc8-0d4f493daf0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:00:41.575599 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-7dd7bf31-ee26-4900-9aa2-16ed35c9d0d7 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:00:41.576917 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-7dd7bf31-ee26-4900-9aa2-16ed35c9d0d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:00:41.580210 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 758/1554] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:00:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 16:00:41.581065 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114aa7b9-3fef-499e-9cc8-0d4f493daf0b None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:00:41.590331 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 797/1555] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:00:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 16:00:41.595015 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114aa7b9-3fef-499e-9cc8-0d4f493daf0b None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:00:41.595852 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9e380f-9a08-4794-94a0-6e7a0691bc75 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:00:41.596157 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9e380f-9a08-4794-94a0-6e7a0691bc75 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:00:41.596446 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9e380f-9a08-4794-94a0-6e7a0691bc75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:00:41.599261 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881825a4-0bb6-4642-9f7a-45e0cbb15008 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:00:41.599963 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881825a4-0bb6-4642-9f7a-45e0cbb15008 admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:00:41.600254 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881825a4-0bb6-4642-9f7a-45e0cbb15008 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:00:41.600591 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881825a4-0bb6-4642-9f7a-45e0cbb15008 admin admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:00:41.601342 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881825a4-0bb6-4642-9f7a-45e0cbb15008 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:00:41.603013 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881825a4-0bb6-4642-9f7a-45e0cbb15008 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:00:41.614439 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 759/1556] 10.222.0.46 () {58 vars in 1121 bytes} [Thu Jun 23 16:00:41 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 16:00:41.615834 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd1a7c7-06e8-4fc2-a954-a168b017cb24 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:00:41.616134 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd1a7c7-06e8-4fc2-a954-a168b017cb24 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:00:41.616328 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-be9e380f-9a08-4794-94a0-6e7a0691bc75 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:00:41.616440 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd1a7c7-06e8-4fc2-a954-a168b017cb24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:00:41.617618 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-be9e380f-9a08-4794-94a0-6e7a0691bc75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:00:41.625829 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 798/1557] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:00:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 16:00:41.627822 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9e380f-9a08-4794-94a0-6e7a0691bc75 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:00:41.635898 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-5bd1a7c7-06e8-4fc2-a954-a168b017cb24 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:00:41.637148 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-5bd1a7c7-06e8-4fc2-a954-a168b017cb24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:00:41.638748 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9e380f-9a08-4794-94a0-6e7a0691bc75 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:00:41.643413 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc0551e-ca01-4a85-9ba4-fbea749dd67b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:00:41.644089 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc0551e-ca01-4a85-9ba4-fbea749dd67b admin admin] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:00:41.644368 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc0551e-ca01-4a85-9ba4-fbea749dd67b admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:00:41.644639 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc0551e-ca01-4a85-9ba4-fbea749dd67b admin admin] PATH_INFO: `/v3/projects` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:00:41.645339 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc0551e-ca01-4a85-9ba4-fbea749dd67b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:00:41.645983 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 760/1558] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:00:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 16:00:41.646831 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd1a7c7-06e8-4fc2-a954-a168b017cb24 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:00:41.647281 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc0551e-ca01-4a85-9ba4-fbea749dd67b admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:00:41.658339 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd1a7c7-06e8-4fc2-a954-a168b017cb24 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:00:41.663627 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cb410c-bb17-44bf-b9c2-2afa1cda8903 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:00:41.664313 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cb410c-bb17-44bf-b9c2-2afa1cda8903 admin admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:00:41.664708 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cb410c-bb17-44bf-b9c2-2afa1cda8903 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:00:41.665033 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cb410c-bb17-44bf-b9c2-2afa1cda8903 admin admin] PATH_INFO: `/v3/projects` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:00:41.665981 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77cb410c-bb17-44bf-b9c2-2afa1cda8903 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:00:41.669206 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77cb410c-bb17-44bf-b9c2-2afa1cda8903 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:00:41.686903 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 799/1559] 10.222.0.46 () {60 vars in 1152 bytes} [Thu Jun 23 16:00:41 2022] POST /identity/v3/projects => generated 403 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 16:00:41.686903 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86983b6e-91dd-4aa1-9bcc-ecca62cd198d None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:00:41.686903 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86983b6e-91dd-4aa1-9bcc-ecca62cd198d None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:00:41.686903 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86983b6e-91dd-4aa1-9bcc-ecca62cd198d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:00:41.703756 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 761/1560] 10.222.0.46 () {60 vars in 1152 bytes} [Thu Jun 23 16:00:41 2022] POST /identity/v3/projects => generated 365 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 16:00:41.703756 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cb410c-bb17-44bf-b9c2-2afa1cda8903 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:00:41.710155 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-86983b6e-91dd-4aa1-9bcc-ecca62cd198d None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:00:41.711743 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-86983b6e-91dd-4aa1-9bcc-ecca62cd198d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:00:41.760269 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:00:41.776658 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cb410c-bb17-44bf-b9c2-2afa1cda8903 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:00:41.784319 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1654555-ab84-4131-b445-83de71fd4c42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:00:41.785363 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1654555-ab84-4131-b445-83de71fd4c42 admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:00:41.785845 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1654555-ab84-4131-b445-83de71fd4c42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:00:41.786314 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1654555-ab84-4131-b445-83de71fd4c42 admin admin] PATH_INFO: `/v3/domains` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:00:41.787909 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1654555-ab84-4131-b445-83de71fd4c42 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 16:00:41.788476 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1654555-ab84-4131-b445-83de71fd4c42 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:00:41.791621 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1654555-ab84-4131-b445-83de71fd4c42 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:00:41.798642 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 800/1561] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:00:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 16:00:41.798642 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e87e572-0926-41ee-860f-8b6b72c7e527 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:00:41.798642 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e87e572-0926-41ee-860f-8b6b72c7e527 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:00:41.798642 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e87e572-0926-41ee-860f-8b6b72c7e527 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:00:41.808252 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:00:41.812381 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 762/1562] 10.222.0.46 () {58 vars in 1152 bytes} [Thu Jun 23 16:00:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 16:00:41.813761 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1654555-ab84-4131-b445-83de71fd4c42 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:00:41.828716 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-4e87e572-0926-41ee-860f-8b6b72c7e527 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:00:41.828988 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1654555-ab84-4131-b445-83de71fd4c42 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:00:41.829946 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-4e87e572-0926-41ee-860f-8b6b72c7e527 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:00:41.834027 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7855463-768b-47cd-83ac-3979e07dbe8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:00:41.834751 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7855463-768b-47cd-83ac-3979e07dbe8c admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:00:41.835045 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7855463-768b-47cd-83ac-3979e07dbe8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:00:41.835347 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7855463-768b-47cd-83ac-3979e07dbe8c admin admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:00:41.836185 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7855463-768b-47cd-83ac-3979e07dbe8c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:00:41.837873 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7855463-768b-47cd-83ac-3979e07dbe8c admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:00:41.839172 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 801/1563] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:00:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 16:00:41.840667 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2bc897-6a6b-4324-b92c-30f9d66d372d None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:00:41.840953 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2bc897-6a6b-4324-b92c-30f9d66d372d None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:00:41.841315 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2bc897-6a6b-4324-b92c-30f9d66d372d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:00:41.863475 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 763/1564] 10.222.0.46 () {58 vars in 1121 bytes} [Thu Jun 23 16:00:41 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 16:00:41.864520 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7855463-768b-47cd-83ac-3979e07dbe8c admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:00:41.876322 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7855463-768b-47cd-83ac-3979e07dbe8c admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:00:41.878053 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-0c2bc897-6a6b-4324-b92c-30f9d66d372d None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:00:41.879515 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-0c2bc897-6a6b-4324-b92c-30f9d66d372d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:00:41.883768 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b6f03a-b799-4ec8-9da5-c931c4dec4c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:00:41.884912 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b6f03a-b799-4ec8-9da5-c931c4dec4c7 admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:00:41.885416 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b6f03a-b799-4ec8-9da5-c931c4dec4c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:00:41.885868 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b6f03a-b799-4ec8-9da5-c931c4dec4c7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:00:41.887029 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b6f03a-b799-4ec8-9da5-c931c4dec4c7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 16:00:41.887577 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b6f03a-b799-4ec8-9da5-c931c4dec4c7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:00:41.888183 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 802/1565] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:00:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 16:00:41.891977 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2bc897-6a6b-4324-b92c-30f9d66d372d None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:00:41.892109 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b6f03a-b799-4ec8-9da5-c931c4dec4c7 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:00:41.899844 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2bc897-6a6b-4324-b92c-30f9d66d372d None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:00:41.906054 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a35dde-511d-4c5a-bab2-ea8102e7144b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:00:41.906853 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a35dde-511d-4c5a-bab2-ea8102e7144b admin admin] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:00:41.907181 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a35dde-511d-4c5a-bab2-ea8102e7144b admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:00:41.907436 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a35dde-511d-4c5a-bab2-ea8102e7144b admin admin] PATH_INFO: `/v3/users` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:00:41.908208 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a35dde-511d-4c5a-bab2-ea8102e7144b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:00:41.910319 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a35dde-511d-4c5a-bab2-ea8102e7144b admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:00:41.910942 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 764/1566] 10.222.0.46 () {58 vars in 1152 bytes} [Thu Jun 23 16:00:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 16:00:41.912092 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b6f03a-b799-4ec8-9da5-c931c4dec4c7 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:00:41.926486 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b6f03a-b799-4ec8-9da5-c931c4dec4c7 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:00:41.933976 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2203f2-986a-4584-b4dd-a8cce6046d6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:00:41.934743 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2203f2-986a-4584-b4dd-a8cce6046d6a admin admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:00:41.935138 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2203f2-986a-4584-b4dd-a8cce6046d6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:00:41.935457 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2203f2-986a-4584-b4dd-a8cce6046d6a admin admin] PATH_INFO: `/v3/users` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:00:41.936138 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2203f2-986a-4584-b4dd-a8cce6046d6a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:00:41.938232 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2203f2-986a-4584-b4dd-a8cce6046d6a admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:00:41.942081 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 803/1567] 10.222.0.46 () {60 vars in 1143 bytes} [Thu Jun 23 16:00:41 2022] POST /identity/v3/users => generated 302 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 16:00:41.942966 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a35dde-511d-4c5a-bab2-ea8102e7144b admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:00:41.956009 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a35dde-511d-4c5a-bab2-ea8102e7144b admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:00:41.961076 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b39c4c-1483-4c0f-97e8-d674ba90c59a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:00:41.961607 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 765/1568] 10.222.0.46 () {60 vars in 1143 bytes} [Thu Jun 23 16:00:41 2022] POST /identity/v3/users => generated 283 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 16:00:41.961824 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b39c4c-1483-4c0f-97e8-d674ba90c59a admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:00:41.962464 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b39c4c-1483-4c0f-97e8-d674ba90c59a admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:00:41.962464 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b39c4c-1483-4c0f-97e8-d674ba90c59a admin admin] PATH_INFO: `/v3/domains` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:00:41.963211 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b39c4c-1483-4c0f-97e8-d674ba90c59a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 16:00:41.963538 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b39c4c-1483-4c0f-97e8-d674ba90c59a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:00:41.965841 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b39c4c-1483-4c0f-97e8-d674ba90c59a admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:00:41.966464 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2203f2-986a-4584-b4dd-a8cce6046d6a admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:00:41.979458 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 804/1569] 10.222.0.46 () {58 vars in 1152 bytes} [Thu Jun 23 16:00:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 16:00:41.980258 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b39c4c-1483-4c0f-97e8-d674ba90c59a admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:00:41.985660 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2203f2-986a-4584-b4dd-a8cce6046d6a admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:00:41.993189 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadbbbec-a489-4999-b92b-8de1290f5ee9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:00:41.993606 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b39c4c-1483-4c0f-97e8-d674ba90c59a admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:00:41.994158 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadbbbec-a489-4999-b92b-8de1290f5ee9 admin admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:00:41.994449 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadbbbec-a489-4999-b92b-8de1290f5ee9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:00:41.994749 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadbbbec-a489-4999-b92b-8de1290f5ee9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:00:41.995918 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadbbbec-a489-4999-b92b-8de1290f5ee9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:00:41.998771 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7449fa-6528-442d-80c1-76b3feaf94d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:00:41.999533 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7449fa-6528-442d-80c1-76b3feaf94d6 admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:00:41.999767 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadbbbec-a489-4999-b92b-8de1290f5ee9 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:00:42.000195 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7449fa-6528-442d-80c1-76b3feaf94d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:00:42.003227 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7449fa-6528-442d-80c1-76b3feaf94d6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:00:42.003227 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7449fa-6528-442d-80c1-76b3feaf94d6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 16:00:42.003227 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7449fa-6528-442d-80c1-76b3feaf94d6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:00:42.003227 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7449fa-6528-442d-80c1-76b3feaf94d6 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:00:42.029494 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 805/1570] 10.222.0.46 () {58 vars in 1152 bytes} [Thu Jun 23 16:00:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 16:00:42.029494 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7449fa-6528-442d-80c1-76b3feaf94d6 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:00:42.041945 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7449fa-6528-442d-80c1-76b3feaf94d6 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:00:42.046736 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fc61b2-2b87-4a93-92f6-38799ed2f660 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:00:42.047663 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fc61b2-2b87-4a93-92f6-38799ed2f660 admin admin] REQUEST_METHOD: `PUT` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:00:42.047788 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 766/1571] 10.222.0.46 () {60 vars in 1152 bytes} [Thu Jun 23 16:00:41 2022] POST /identity/v3/projects => generated 391 bytes in 82 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 16:00:42.047950 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fc61b2-2b87-4a93-92f6-38799ed2f660 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:00:42.048396 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fc61b2-2b87-4a93-92f6-38799ed2f660 admin admin] PATH_INFO: `/v3/projects/27aed572a9da43ca919d77bd579224a9/users/82e02e90305a424c8e719a92bd084a5f/roles/e2a5bbf3200a4d4091d72bc7ff78c34d` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:00:42.048563 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadbbbec-a489-4999-b92b-8de1290f5ee9 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:00:42.069170 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fc61b2-2b87-4a93-92f6-38799ed2f660 admin admin] RBAC: Authorizing `identity:create_grant(project_id=27aed572a9da43ca919d77bd579224a9, user_id=82e02e90305a424c8e719a92bd084a5f, role_id=e2a5bbf3200a4d4091d72bc7ff78c34d)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:00:42.071943 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fc61b2-2b87-4a93-92f6-38799ed2f660 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:00:42.092923 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 806/1572] 10.222.0.46 () {60 vars in 1482 bytes} [Thu Jun 23 16:00:42 2022] PUT /identity/v3/projects/27aed572a9da43ca919d77bd579224a9/users/82e02e90305a424c8e719a92bd084a5f/roles/e2a5bbf3200a4d4091d72bc7ff78c34d => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 16:00:42.094286 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fc61b2-2b87-4a93-92f6-38799ed2f660 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:00:42.135347 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadbbbec-a489-4999-b92b-8de1290f5ee9 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:00:42.169007 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fc61b2-2b87-4a93-92f6-38799ed2f660 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:00:42.174449 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5191a7-7dff-4e93-8d4a-8d8b3a347145 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:00:42.175177 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5191a7-7dff-4e93-8d4a-8d8b3a347145 admin admin] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:00:42.175464 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5191a7-7dff-4e93-8d4a-8d8b3a347145 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:00:42.175748 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5191a7-7dff-4e93-8d4a-8d8b3a347145 admin admin] PATH_INFO: `/v3/projects` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:00:42.176468 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5191a7-7dff-4e93-8d4a-8d8b3a347145 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:00:42.178488 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5191a7-7dff-4e93-8d4a-8d8b3a347145 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:00:42.208385 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 807/1573] 10.222.0.46 () {60 vars in 1152 bytes} [Thu Jun 23 16:00:42 2022] POST /identity/v3/projects => generated 383 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 16:00:42.213485 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40109b5d-ed0d-4123-9277-772016edc4d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:00:42.214279 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5191a7-7dff-4e93-8d4a-8d8b3a347145 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:00:42.214434 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40109b5d-ed0d-4123-9277-772016edc4d0 admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:00:42.214434 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40109b5d-ed0d-4123-9277-772016edc4d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:00:42.214434 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40109b5d-ed0d-4123-9277-772016edc4d0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:00:42.214434 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40109b5d-ed0d-4123-9277-772016edc4d0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 16:00:42.214434 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40109b5d-ed0d-4123-9277-772016edc4d0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:00:42.215664 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40109b5d-ed0d-4123-9277-772016edc4d0 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:00:42.239472 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 767/1574] 10.222.0.46 () {58 vars in 1152 bytes} [Thu Jun 23 16:00:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 16:00:42.240868 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40109b5d-ed0d-4123-9277-772016edc4d0 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:00:42.259305 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40109b5d-ed0d-4123-9277-772016edc4d0 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:00:42.290213 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:00:42.305768 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5191a7-7dff-4e93-8d4a-8d8b3a347145 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:00:42.314783 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085408ae-59a7-481e-bf16-eb3b55c9eebc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:00:42.315879 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085408ae-59a7-481e-bf16-eb3b55c9eebc admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:00:42.316376 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085408ae-59a7-481e-bf16-eb3b55c9eebc admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:00:42.316871 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085408ae-59a7-481e-bf16-eb3b55c9eebc admin admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:00:42.318254 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085408ae-59a7-481e-bf16-eb3b55c9eebc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:00:42.320765 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085408ae-59a7-481e-bf16-eb3b55c9eebc admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:00:42.333657 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 808/1575] 10.222.0.46 () {58 vars in 1121 bytes} [Thu Jun 23 16:00:42 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 16:00:42.334475 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085408ae-59a7-481e-bf16-eb3b55c9eebc admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:00:42.340512 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:00:42.342599 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee6a2f5-60ca-4f2f-ab6e-eafcc8f636dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:00:42.343181 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee6a2f5-60ca-4f2f-ab6e-eafcc8f636dc admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:00:42.343682 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee6a2f5-60ca-4f2f-ab6e-eafcc8f636dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:00:42.344178 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee6a2f5-60ca-4f2f-ab6e-eafcc8f636dc admin admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:00:42.345616 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee6a2f5-60ca-4f2f-ab6e-eafcc8f636dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:00:42.346884 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085408ae-59a7-481e-bf16-eb3b55c9eebc admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:00:42.348650 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee6a2f5-60ca-4f2f-ab6e-eafcc8f636dc admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:00:42.353360 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea072e2-9e98-4335-9717-25f44d2399e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:00:42.354032 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea072e2-9e98-4335-9717-25f44d2399e7 admin admin] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:00:42.354350 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea072e2-9e98-4335-9717-25f44d2399e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:00:42.354633 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea072e2-9e98-4335-9717-25f44d2399e7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:00:42.355359 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea072e2-9e98-4335-9717-25f44d2399e7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:00:42.357310 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea072e2-9e98-4335-9717-25f44d2399e7 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:00:42.364736 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 768/1576] 10.222.0.46 () {58 vars in 1121 bytes} [Thu Jun 23 16:00:42 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 16:00:42.367827 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee6a2f5-60ca-4f2f-ab6e-eafcc8f636dc admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:00:42.379523 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee6a2f5-60ca-4f2f-ab6e-eafcc8f636dc admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:00:42.386556 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e695c45-2b8f-4bc8-ad00-a6f567fef6cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:00:42.387764 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e695c45-2b8f-4bc8-ad00-a6f567fef6cb admin admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:00:42.388271 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e695c45-2b8f-4bc8-ad00-a6f567fef6cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:00:42.388788 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e695c45-2b8f-4bc8-ad00-a6f567fef6cb admin admin] PATH_INFO: `/v3/projects` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:00:42.390253 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e695c45-2b8f-4bc8-ad00-a6f567fef6cb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:00:42.394073 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e695c45-2b8f-4bc8-ad00-a6f567fef6cb admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:00:42.399502 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 809/1577] 10.222.0.46 () {60 vars in 1152 bytes} [Thu Jun 23 16:00:42 2022] POST /identity/v3/projects => generated 385 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 16:00:42.400766 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea072e2-9e98-4335-9717-25f44d2399e7 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:00:42.447647 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 769/1578] 10.222.0.46 () {60 vars in 1152 bytes} [Thu Jun 23 16:00:42 2022] POST /identity/v3/projects => generated 381 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 16:00:42.447647 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f627c039-2b04-4adf-8ff9-cb6d5c988e40 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:00:42.447647 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f627c039-2b04-4adf-8ff9-cb6d5c988e40 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:00:42.447647 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f627c039-2b04-4adf-8ff9-cb6d5c988e40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:00:42.467894 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-f627c039-2b04-4adf-8ff9-cb6d5c988e40 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:00:42.469178 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-f627c039-2b04-4adf-8ff9-cb6d5c988e40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:00:42.486437 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea072e2-9e98-4335-9717-25f44d2399e7 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:00:42.547391 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 770/1579] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:00:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 16:00:42.548287 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f627c039-2b04-4adf-8ff9-cb6d5c988e40 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:00:42.561619 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f627c039-2b04-4adf-8ff9-cb6d5c988e40 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:00:42.567337 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1920990c-666c-4370-b391-b2458762e7ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:00:42.568044 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1920990c-666c-4370-b391-b2458762e7ff admin admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:00:42.568354 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1920990c-666c-4370-b391-b2458762e7ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:00:42.568639 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1920990c-666c-4370-b391-b2458762e7ff admin admin] PATH_INFO: `/v3/users` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:00:42.569362 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1920990c-666c-4370-b391-b2458762e7ff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:00:42.575390 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1920990c-666c-4370-b391-b2458762e7ff admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:00:42.576102 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ac2050-938a-426a-9e68-93d5c350a83b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:00:42.576886 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ac2050-938a-426a-9e68-93d5c350a83b admin admin] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:00:42.577491 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ac2050-938a-426a-9e68-93d5c350a83b admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:00:42.577894 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ac2050-938a-426a-9e68-93d5c350a83b admin admin] PATH_INFO: `/v3/users` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:00:42.579167 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ac2050-938a-426a-9e68-93d5c350a83b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:00:42.581630 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ac2050-938a-426a-9e68-93d5c350a83b admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:00:42.597980 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 771/1580] 10.222.0.46 () {60 vars in 1143 bytes} [Thu Jun 23 16:00:42 2022] POST /identity/v3/users => generated 292 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 16:00:42.599155 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1920990c-666c-4370-b391-b2458762e7ff admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:00:42.611889 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1920990c-666c-4370-b391-b2458762e7ff admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:00:42.618406 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 810/1581] 10.222.0.46 () {60 vars in 1143 bytes} [Thu Jun 23 16:00:42 2022] POST /identity/v3/users => generated 296 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 16:00:42.618406 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ac2050-938a-426a-9e68-93d5c350a83b admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:00:42.619560 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e98ad0-80db-46e0-8bf0-97a4a85b8dd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:00:42.620899 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e98ad0-80db-46e0-8bf0-97a4a85b8dd4 admin admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:00:42.621190 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e98ad0-80db-46e0-8bf0-97a4a85b8dd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:00:42.621473 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e98ad0-80db-46e0-8bf0-97a4a85b8dd4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:00:42.622183 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e98ad0-80db-46e0-8bf0-97a4a85b8dd4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:00:42.624187 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e98ad0-80db-46e0-8bf0-97a4a85b8dd4 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:00:42.630287 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ac2050-938a-426a-9e68-93d5c350a83b admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:00:42.635178 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d502f210-a9c4-486e-a6f0-d419d28dffe4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:00:42.636480 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d502f210-a9c4-486e-a6f0-d419d28dffe4 admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:00:42.636756 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d502f210-a9c4-486e-a6f0-d419d28dffe4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:00:42.637020 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d502f210-a9c4-486e-a6f0-d419d28dffe4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:00:42.637796 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d502f210-a9c4-486e-a6f0-d419d28dffe4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:00:42.639450 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d502f210-a9c4-486e-a6f0-d419d28dffe4 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:00:42.651169 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 811/1582] 10.222.0.46 () {58 vars in 1121 bytes} [Thu Jun 23 16:00:42 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 16:00:42.651169 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d502f210-a9c4-486e-a6f0-d419d28dffe4 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:00:42.660372 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 772/1583] 10.222.0.46 () {60 vars in 1152 bytes} [Thu Jun 23 16:00:42 2022] POST /identity/v3/projects => generated 387 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 16:00:42.661222 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e98ad0-80db-46e0-8bf0-97a4a85b8dd4 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:00:42.672732 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d502f210-a9c4-486e-a6f0-d419d28dffe4 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:00:42.745770 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9379cb1-2b8c-47bc-8ef9-fe73b384d5c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:00:42.746379 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9379cb1-2b8c-47bc-8ef9-fe73b384d5c8 admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:00:42.746684 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9379cb1-2b8c-47bc-8ef9-fe73b384d5c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:00:42.746984 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9379cb1-2b8c-47bc-8ef9-fe73b384d5c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:00:42.747722 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9379cb1-2b8c-47bc-8ef9-fe73b384d5c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:00:42.749419 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9379cb1-2b8c-47bc-8ef9-fe73b384d5c8 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:00:42.752138 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e98ad0-80db-46e0-8bf0-97a4a85b8dd4 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:00:42.760145 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28f43d6-c05b-43b1-9f54-0ebc2e6a79bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:00:42.761008 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 812/1584] 10.222.0.46 () {58 vars in 1121 bytes} [Thu Jun 23 16:00:42 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 16:00:42.761399 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28f43d6-c05b-43b1-9f54-0ebc2e6a79bd admin admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:00:42.761687 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9379cb1-2b8c-47bc-8ef9-fe73b384d5c8 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:00:42.761922 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28f43d6-c05b-43b1-9f54-0ebc2e6a79bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:00:42.762426 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28f43d6-c05b-43b1-9f54-0ebc2e6a79bd admin admin] PATH_INFO: `/v3/users` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:00:42.763746 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28f43d6-c05b-43b1-9f54-0ebc2e6a79bd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:00:42.767323 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28f43d6-c05b-43b1-9f54-0ebc2e6a79bd admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:00:42.769401 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:00:42.777030 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9379cb1-2b8c-47bc-8ef9-fe73b384d5c8 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:00:42.778930 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:00:42.783044 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff7b1bf-0167-440b-9ed2-ac607ff80205 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:00:42.783757 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff7b1bf-0167-440b-9ed2-ac607ff80205 admin admin] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:00:42.784043 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff7b1bf-0167-440b-9ed2-ac607ff80205 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:00:42.784319 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff7b1bf-0167-440b-9ed2-ac607ff80205 admin admin] PATH_INFO: `/v3/users` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:00:42.785023 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff7b1bf-0167-440b-9ed2-ac607ff80205 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:00:42.787011 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff7b1bf-0167-440b-9ed2-ac607ff80205 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:00:42.804138 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 773/1585] 10.222.0.46 () {60 vars in 1143 bytes} [Thu Jun 23 16:00:42 2022] POST /identity/v3/users => generated 293 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 16:00:42.805028 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28f43d6-c05b-43b1-9f54-0ebc2e6a79bd admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:00:42.814626 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 813/1586] 10.222.0.46 () {60 vars in 1143 bytes} [Thu Jun 23 16:00:42 2022] POST /identity/v3/users => generated 291 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 16:00:42.815508 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff7b1bf-0167-440b-9ed2-ac607ff80205 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:00:42.829408 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff7b1bf-0167-440b-9ed2-ac607ff80205 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:00:42.835632 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b532ee5a-a6ba-4c7c-8467-702964b9e611 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:00:42.836831 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b532ee5a-a6ba-4c7c-8467-702964b9e611 admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:00:42.837331 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b532ee5a-a6ba-4c7c-8467-702964b9e611 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:00:42.837712 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b532ee5a-a6ba-4c7c-8467-702964b9e611 admin admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:00:42.839386 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b532ee5a-a6ba-4c7c-8467-702964b9e611 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:00:42.842781 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b532ee5a-a6ba-4c7c-8467-702964b9e611 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:00:42.870633 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 814/1587] 10.222.0.46 () {58 vars in 1121 bytes} [Thu Jun 23 16:00:42 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 16:00:42.872270 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b532ee5a-a6ba-4c7c-8467-702964b9e611 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:00:42.883272 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b532ee5a-a6ba-4c7c-8467-702964b9e611 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:00:42.884135 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28f43d6-c05b-43b1-9f54-0ebc2e6a79bd admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:00:42.888540 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42650c6-7479-423c-84e2-3ff53dc6a862 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51784064951a4619ae09b9167fb0fd0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:00:42.888702 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329f6546-20ff-497a-b8e1-e0346a44c2d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:00:42.889215 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42650c6-7479-423c-84e2-3ff53dc6a862 service neutron] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:00:42.889328 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329f6546-20ff-497a-b8e1-e0346a44c2d1 admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:00:42.889538 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42650c6-7479-423c-84e2-3ff53dc6a862 service neutron] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:00:42.889644 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329f6546-20ff-497a-b8e1-e0346a44c2d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:00:42.889815 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329f6546-20ff-497a-b8e1-e0346a44c2d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:00:42.889992 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42650c6-7479-423c-84e2-3ff53dc6a862 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:00:42.890568 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329f6546-20ff-497a-b8e1-e0346a44c2d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:00:42.892254 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329f6546-20ff-497a-b8e1-e0346a44c2d1 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:00:42.902505 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42650c6-7479-423c-84e2-3ff53dc6a862 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:00:42.903704 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 815/1588] 10.222.0.46 () {58 vars in 1121 bytes} [Thu Jun 23 16:00:42 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 16:00:42.904058 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42650c6-7479-423c-84e2-3ff53dc6a862 service neutron] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:00:42.907648 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329f6546-20ff-497a-b8e1-e0346a44c2d1 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:00:42.914524 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 774/1589] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jun 23 16:00:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 16:00:42.915421 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42650c6-7479-423c-84e2-3ff53dc6a862 service neutron] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:00:42.923143 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329f6546-20ff-497a-b8e1-e0346a44c2d1 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:00:42.928077 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42650c6-7479-423c-84e2-3ff53dc6a862 service neutron] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:00:42.928430 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b9f9ae-c8c6-409a-9207-7d0452b619ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:00:42.929249 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b9f9ae-c8c6-409a-9207-7d0452b619ba admin admin] REQUEST_METHOD: `PUT` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:00:42.929544 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b9f9ae-c8c6-409a-9207-7d0452b619ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:00:42.929820 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b9f9ae-c8c6-409a-9207-7d0452b619ba admin admin] PATH_INFO: `/v3/projects/d91ab0ef24e84c07ad4948303d1f184c/users/743784269ad54fe295ada341b492739e/roles/e2a5bbf3200a4d4091d72bc7ff78c34d` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:00:42.935267 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a975242-fe23-4e3f-a8f8-3d83a51cb2df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:00:42.936332 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a975242-fe23-4e3f-a8f8-3d83a51cb2df admin admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:00:42.936821 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a975242-fe23-4e3f-a8f8-3d83a51cb2df admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:00:42.937240 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a975242-fe23-4e3f-a8f8-3d83a51cb2df admin admin] PATH_INFO: `/v3/users` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:00:42.938140 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a975242-fe23-4e3f-a8f8-3d83a51cb2df admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:00:42.941225 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a975242-fe23-4e3f-a8f8-3d83a51cb2df admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:00:42.960984 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b9f9ae-c8c6-409a-9207-7d0452b619ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=d91ab0ef24e84c07ad4948303d1f184c, user_id=743784269ad54fe295ada341b492739e, role_id=e2a5bbf3200a4d4091d72bc7ff78c34d)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:00:42.967021 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b9f9ae-c8c6-409a-9207-7d0452b619ba admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:00:42.971288 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 775/1590] 10.222.0.46 () {60 vars in 1143 bytes} [Thu Jun 23 16:00:42 2022] POST /identity/v3/users => generated 294 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 16:00:42.971288 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a975242-fe23-4e3f-a8f8-3d83a51cb2df admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:00:42.986904 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a975242-fe23-4e3f-a8f8-3d83a51cb2df admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:00:42.991384 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 816/1591] 10.222.0.46 () {60 vars in 1482 bytes} [Thu Jun 23 16:00:42 2022] PUT /identity/v3/projects/d91ab0ef24e84c07ad4948303d1f184c/users/743784269ad54fe295ada341b492739e/roles/e2a5bbf3200a4d4091d72bc7ff78c34d => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 16:00:42.992365 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b9f9ae-c8c6-409a-9207-7d0452b619ba admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:00:43.074601 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b5dd48-65ee-4e99-82f5-b702e7fb1831 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:00:43.076023 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b5dd48-65ee-4e99-82f5-b702e7fb1831 admin admin] REQUEST_METHOD: `PUT` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:00:43.076304 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b5dd48-65ee-4e99-82f5-b702e7fb1831 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:00:43.076549 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b5dd48-65ee-4e99-82f5-b702e7fb1831 admin admin] PATH_INFO: `/v3/projects/fe4cb80c54f3454a897675c905ef1732/users/3536ffe4f5194e78b4edee59471752b0/roles/e2a5bbf3200a4d4091d72bc7ff78c34d` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:00:43.100387 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b5dd48-65ee-4e99-82f5-b702e7fb1831 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe4cb80c54f3454a897675c905ef1732, user_id=3536ffe4f5194e78b4edee59471752b0, role_id=e2a5bbf3200a4d4091d72bc7ff78c34d)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:00:43.102440 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: WARNING py.warnings [None req-97b5dd48-65ee-4e99-82f5-b702e7fb1831 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 23 16:00:43.102440 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: warnings.warn(msg) Jun 23 16:00:43.102440 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: Jun 23 16:00:43.106717 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b5dd48-65ee-4e99-82f5-b702e7fb1831 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:00:43.108744 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b9f9ae-c8c6-409a-9207-7d0452b619ba admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:00:43.116406 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0056fe35-7077-42f4-a900-f4a5c8509890 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:00:43.116406 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0056fe35-7077-42f4-a900-f4a5c8509890 admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:00:43.116406 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0056fe35-7077-42f4-a900-f4a5c8509890 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:00:43.116406 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0056fe35-7077-42f4-a900-f4a5c8509890 admin admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:00:43.117440 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0056fe35-7077-42f4-a900-f4a5c8509890 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:00:43.119214 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0056fe35-7077-42f4-a900-f4a5c8509890 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:00:43.139781 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 776/1592] 10.222.0.46 () {60 vars in 1482 bytes} [Thu Jun 23 16:00:42 2022] PUT /identity/v3/projects/fe4cb80c54f3454a897675c905ef1732/users/3536ffe4f5194e78b4edee59471752b0/roles/e2a5bbf3200a4d4091d72bc7ff78c34d => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 16:00:43.140681 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 817/1593] 10.222.0.46 () {58 vars in 1121 bytes} [Thu Jun 23 16:00:42 2022] GET /identity/v3/roles => generated 1420 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 16:00:43.141529 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0056fe35-7077-42f4-a900-f4a5c8509890 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:00:43.145550 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b5dd48-65ee-4e99-82f5-b702e7fb1831 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:00:43.154539 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0056fe35-7077-42f4-a900-f4a5c8509890 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:00:43.247396 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1295ce0b-0478-40fa-b982-f1e75d40a98a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:00:43.248220 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1295ce0b-0478-40fa-b982-f1e75d40a98a admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:00:43.248568 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1295ce0b-0478-40fa-b982-f1e75d40a98a admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:00:43.249041 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1295ce0b-0478-40fa-b982-f1e75d40a98a admin admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:00:43.249911 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1295ce0b-0478-40fa-b982-f1e75d40a98a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:00:43.251250 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:INSERT=7,keystone:SELECT=93 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:00:43.251907 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1295ce0b-0478-40fa-b982-f1e75d40a98a admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:00:43.258019 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b5dd48-65ee-4e99-82f5-b702e7fb1831 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:00:43.272229 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 818/1594] 10.222.0.46 () {58 vars in 1121 bytes} [Thu Jun 23 16:00:43 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 16:00:43.273659 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1295ce0b-0478-40fa-b982-f1e75d40a98a admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:00:43.284678 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69315a3-9889-40e6-9105-a3c18ac05ad1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:00:43.285521 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69315a3-9889-40e6-9105-a3c18ac05ad1 admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:00:43.285877 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69315a3-9889-40e6-9105-a3c18ac05ad1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:00:43.286286 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69315a3-9889-40e6-9105-a3c18ac05ad1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:00:43.287224 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69315a3-9889-40e6-9105-a3c18ac05ad1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:00:43.298642 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69315a3-9889-40e6-9105-a3c18ac05ad1 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:00:43.299570 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1295ce0b-0478-40fa-b982-f1e75d40a98a admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:00:43.307764 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d76892f-beae-4182-89cc-94545c8c6dcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:00:43.308864 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d76892f-beae-4182-89cc-94545c8c6dcf admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:00:43.309651 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d76892f-beae-4182-89cc-94545c8c6dcf admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:00:43.309830 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d76892f-beae-4182-89cc-94545c8c6dcf admin admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:00:43.311469 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d76892f-beae-4182-89cc-94545c8c6dcf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:00:43.314805 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 777/1595] 10.222.0.46 () {58 vars in 1121 bytes} [Thu Jun 23 16:00:43 2022] GET /identity/v3/roles => generated 1420 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 16:00:43.314805 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69315a3-9889-40e6-9105-a3c18ac05ad1 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:00:43.315095 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d76892f-beae-4182-89cc-94545c8c6dcf admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:00:43.328975 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 819/1596] 10.222.0.46 () {58 vars in 1121 bytes} [Thu Jun 23 16:00:43 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 16:00:43.330215 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5902393-6364-4a5f-b3a7-7519e6de2d56 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:00:43.330665 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5902393-6364-4a5f-b3a7-7519e6de2d56 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:00:43.330973 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5902393-6364-4a5f-b3a7-7519e6de2d56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:00:43.333642 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69315a3-9889-40e6-9105-a3c18ac05ad1 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:00:43.346776 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84606f5-83c9-4669-ba00-2e6ec582d65f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:00:43.349276 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84606f5-83c9-4669-ba00-2e6ec582d65f admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:00:43.349798 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84606f5-83c9-4669-ba00-2e6ec582d65f admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:00:43.351573 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84606f5-83c9-4669-ba00-2e6ec582d65f admin admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:00:43.353291 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84606f5-83c9-4669-ba00-2e6ec582d65f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:00:43.357173 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84606f5-83c9-4669-ba00-2e6ec582d65f admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:00:43.375343 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 778/1597] 10.222.0.46 () {58 vars in 1121 bytes} [Thu Jun 23 16:00:43 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 16:00:43.375578 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-d5902393-6364-4a5f-b3a7-7519e6de2d56 None None] MFA Rules not processed for user `743784269ad54fe295ada341b492739e`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:00:43.376937 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84606f5-83c9-4669-ba00-2e6ec582d65f admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:00:43.377511 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-d5902393-6364-4a5f-b3a7-7519e6de2d56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:00:43.391872 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84606f5-83c9-4669-ba00-2e6ec582d65f admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:00:43.395986 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:00:43.398354 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d65b475-9918-4477-abda-a2814062744a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:00:43.398681 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d65b475-9918-4477-abda-a2814062744a admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:00:43.399066 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d65b475-9918-4477-abda-a2814062744a admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:00:43.399805 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d65b475-9918-4477-abda-a2814062744a admin admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:00:43.401566 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d65b475-9918-4477-abda-a2814062744a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:00:43.403708 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d65b475-9918-4477-abda-a2814062744a admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:00:43.416325 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 779/1598] 10.222.0.46 () {58 vars in 1121 bytes} [Thu Jun 23 16:00:43 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 16:00:43.422037 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6915f688-8c41-4221-9112-6d2db89bdecf None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:00:43.422409 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6915f688-8c41-4221-9112-6d2db89bdecf None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:00:43.422668 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6915f688-8c41-4221-9112-6d2db89bdecf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:00:43.457938 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-6915f688-8c41-4221-9112-6d2db89bdecf None None] MFA Rules not processed for user `3536ffe4f5194e78b4edee59471752b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:00:43.459806 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-6915f688-8c41-4221-9112-6d2db89bdecf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:00:43.501745 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 820/1599] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:00:43 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 16:00:43.502864 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5902393-6364-4a5f-b3a7-7519e6de2d56 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:00:43.515543 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5902393-6364-4a5f-b3a7-7519e6de2d56 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:00:43.520459 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156c1dc9-42a1-4bd6-bb47-37ff940e27da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:00:43.521470 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156c1dc9-42a1-4bd6-bb47-37ff940e27da admin admin] REQUEST_METHOD: `PUT` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:00:43.521847 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156c1dc9-42a1-4bd6-bb47-37ff940e27da admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:00:43.522197 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156c1dc9-42a1-4bd6-bb47-37ff940e27da admin admin] PATH_INFO: `/v3/projects/80d7b052110b4d61afa8de89a996b580/users/d887ce5851b5410094f1bd83120935c1/roles/e2a5bbf3200a4d4091d72bc7ff78c34d` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:00:43.540750 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156c1dc9-42a1-4bd6-bb47-37ff940e27da admin admin] RBAC: Authorizing `identity:create_grant(project_id=80d7b052110b4d61afa8de89a996b580, user_id=d887ce5851b5410094f1bd83120935c1, role_id=e2a5bbf3200a4d4091d72bc7ff78c34d)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:00:43.543902 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156c1dc9-42a1-4bd6-bb47-37ff940e27da admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:00:43.547049 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 780/1600] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:00:43 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 16:00:43.559894 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6915f688-8c41-4221-9112-6d2db89bdecf None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:00:43.570915 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 821/1601] 10.222.0.46 () {60 vars in 1482 bytes} [Thu Jun 23 16:00:43 2022] PUT /identity/v3/projects/80d7b052110b4d61afa8de89a996b580/users/d887ce5851b5410094f1bd83120935c1/roles/e2a5bbf3200a4d4091d72bc7ff78c34d => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 16:00:43.570915 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156c1dc9-42a1-4bd6-bb47-37ff940e27da admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:00:43.582083 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6915f688-8c41-4221-9112-6d2db89bdecf None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:00:43.676974 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209f5790-da8e-4019-96b6-2a483b199a50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:00:43.679379 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156c1dc9-42a1-4bd6-bb47-37ff940e27da admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:00:43.684265 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209f5790-da8e-4019-96b6-2a483b199a50 admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:00:43.684652 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974f3bef-2640-4c2d-a16d-8abe2d087b17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:00:43.685404 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209f5790-da8e-4019-96b6-2a483b199a50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:00:43.685528 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974f3bef-2640-4c2d-a16d-8abe2d087b17 admin admin] REQUEST_METHOD: `PUT` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:00:43.685825 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974f3bef-2640-4c2d-a16d-8abe2d087b17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:00:43.686091 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974f3bef-2640-4c2d-a16d-8abe2d087b17 admin admin] PATH_INFO: `/v3/projects/f9eeca1030964294b8d813812abdcc5b/users/228e5acf4578494ab1a28e4f530233f2/roles/e2a5bbf3200a4d4091d72bc7ff78c34d` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:00:43.688000 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209f5790-da8e-4019-96b6-2a483b199a50 admin admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:00:43.690089 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209f5790-da8e-4019-96b6-2a483b199a50 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:00:43.692413 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209f5790-da8e-4019-96b6-2a483b199a50 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:00:43.706121 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 781/1602] 10.222.0.46 () {58 vars in 1121 bytes} [Thu Jun 23 16:00:43 2022] GET /identity/v3/roles => generated 1420 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 16:00:43.707219 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209f5790-da8e-4019-96b6-2a483b199a50 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:00:43.713579 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974f3bef-2640-4c2d-a16d-8abe2d087b17 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f9eeca1030964294b8d813812abdcc5b, user_id=228e5acf4578494ab1a28e4f530233f2, role_id=e2a5bbf3200a4d4091d72bc7ff78c34d)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:00:43.717418 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974f3bef-2640-4c2d-a16d-8abe2d087b17 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:00:43.719549 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209f5790-da8e-4019-96b6-2a483b199a50 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:00:43.725082 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49701bd7-57dc-475c-8c6e-d842139b3821 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:00:43.726064 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49701bd7-57dc-475c-8c6e-d842139b3821 admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:00:43.726363 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49701bd7-57dc-475c-8c6e-d842139b3821 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:00:43.726874 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49701bd7-57dc-475c-8c6e-d842139b3821 admin admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:00:43.727789 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49701bd7-57dc-475c-8c6e-d842139b3821 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:00:43.729867 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49701bd7-57dc-475c-8c6e-d842139b3821 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:00:43.744759 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 822/1603] 10.222.0.46 () {60 vars in 1482 bytes} [Thu Jun 23 16:00:43 2022] PUT /identity/v3/projects/f9eeca1030964294b8d813812abdcc5b/users/228e5acf4578494ab1a28e4f530233f2/roles/e2a5bbf3200a4d4091d72bc7ff78c34d => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 16:00:43.744989 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 782/1604] 10.222.0.46 () {58 vars in 1121 bytes} [Thu Jun 23 16:00:43 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 16:00:43.744989 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49701bd7-57dc-475c-8c6e-d842139b3821 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:00:43.748281 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974f3bef-2640-4c2d-a16d-8abe2d087b17 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:00:43.760006 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49701bd7-57dc-475c-8c6e-d842139b3821 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:00:43.833616 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:00:43.842225 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974f3bef-2640-4c2d-a16d-8abe2d087b17 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:00:43.848628 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe4258c-7c11-45ce-afe4-d4cbf24a9f24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:00:43.849852 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe4258c-7c11-45ce-afe4-d4cbf24a9f24 admin admin] REQUEST_METHOD: `PUT` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:00:43.850201 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe4258c-7c11-45ce-afe4-d4cbf24a9f24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:00:43.850628 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe4258c-7c11-45ce-afe4-d4cbf24a9f24 admin admin] PATH_INFO: `/v3/projects/2063fb24abb5408794361a9032e56d3f/users/dac5e323b42f41859c85709a2b49623b/roles/e2a5bbf3200a4d4091d72bc7ff78c34d` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:00:43.854837 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b489cd-0712-4f01-a536-931b38fa3316 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:00:43.855833 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b489cd-0712-4f01-a536-931b38fa3316 admin admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:00:43.856341 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b489cd-0712-4f01-a536-931b38fa3316 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:00:43.856917 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b489cd-0712-4f01-a536-931b38fa3316 admin admin] PATH_INFO: `/v3/projects` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:00:43.857919 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b489cd-0712-4f01-a536-931b38fa3316 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:00:43.860189 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b489cd-0712-4f01-a536-931b38fa3316 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:00:43.875771 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:00:43.882755 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe4258c-7c11-45ce-afe4-d4cbf24a9f24 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2063fb24abb5408794361a9032e56d3f, user_id=dac5e323b42f41859c85709a2b49623b, role_id=e2a5bbf3200a4d4091d72bc7ff78c34d)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:00:43.886669 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe4258c-7c11-45ce-afe4-d4cbf24a9f24 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:00:43.906001 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 823/1605] 10.222.0.46 () {60 vars in 1482 bytes} [Thu Jun 23 16:00:43 2022] PUT /identity/v3/projects/2063fb24abb5408794361a9032e56d3f/users/dac5e323b42f41859c85709a2b49623b/roles/e2a5bbf3200a4d4091d72bc7ff78c34d => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 16:00:43.907253 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe4258c-7c11-45ce-afe4-d4cbf24a9f24 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:00:43.909221 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 783/1606] 10.222.0.46 () {60 vars in 1152 bytes} [Thu Jun 23 16:00:43 2022] POST /identity/v3/projects => generated 403 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 16:00:43.911523 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de51146-6d58-49aa-882c-dc593c4c77a6 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:00:43.911828 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de51146-6d58-49aa-882c-dc593c4c77a6 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:00:43.912444 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de51146-6d58-49aa-882c-dc593c4c77a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:00:43.963256 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-4de51146-6d58-49aa-882c-dc593c4c77a6 None None] MFA Rules not processed for user `d887ce5851b5410094f1bd83120935c1`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:00:43.965055 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-4de51146-6d58-49aa-882c-dc593c4c77a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:00:44.007816 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe4258c-7c11-45ce-afe4-d4cbf24a9f24 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:00:44.069382 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 784/1607] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:00:43 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 16:00:44.071529 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de51146-6d58-49aa-882c-dc593c4c77a6 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:00:44.103852 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7294148-f086-47fd-829e-5bf299fb5e00 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce18eaaba3a6477684fff1fd98ec8b51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:00:44.106714 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7294148-f086-47fd-829e-5bf299fb5e00 service cinder] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:00:44.107193 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7294148-f086-47fd-829e-5bf299fb5e00 service cinder] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:00:44.107623 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7294148-f086-47fd-829e-5bf299fb5e00 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:00:44.211845 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7294148-f086-47fd-829e-5bf299fb5e00 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:00:44.213532 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7294148-f086-47fd-829e-5bf299fb5e00 service cinder] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:00:44.215214 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de51146-6d58-49aa-882c-dc593c4c77a6 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:00:44.221808 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395b7351-509b-497b-a076-3792031642de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:00:44.223382 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395b7351-509b-497b-a076-3792031642de admin admin] REQUEST_METHOD: `PUT` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:00:44.223812 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395b7351-509b-497b-a076-3792031642de admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:00:44.224415 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395b7351-509b-497b-a076-3792031642de admin admin] PATH_INFO: `/v3/projects/e65b4e064fd6448896fb26b9ad2e85ac/users/3263967dbd554d7bb70c8c0b074ff247/roles/e2a5bbf3200a4d4091d72bc7ff78c34d` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:00:44.247853 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 824/1608] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jun 23 16:00:43 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 16:00:44.249775 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffae8abd-83e8-4470-9d3e-95f217a99a1c None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:00:44.250066 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffae8abd-83e8-4470-9d3e-95f217a99a1c None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:00:44.250395 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffae8abd-83e8-4470-9d3e-95f217a99a1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:00:44.250703 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395b7351-509b-497b-a076-3792031642de admin admin] RBAC: Authorizing `identity:create_grant(project_id=e65b4e064fd6448896fb26b9ad2e85ac, user_id=3263967dbd554d7bb70c8c0b074ff247, role_id=e2a5bbf3200a4d4091d72bc7ff78c34d)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:00:44.253856 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395b7351-509b-497b-a076-3792031642de admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:00:44.276677 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 785/1609] 10.222.0.46 () {60 vars in 1482 bytes} [Thu Jun 23 16:00:44 2022] PUT /identity/v3/projects/e65b4e064fd6448896fb26b9ad2e85ac/users/3263967dbd554d7bb70c8c0b074ff247/roles/e2a5bbf3200a4d4091d72bc7ff78c34d => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 16:00:44.277776 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395b7351-509b-497b-a076-3792031642de admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:00:44.293966 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-ffae8abd-83e8-4470-9d3e-95f217a99a1c None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:00:44.297725 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-ffae8abd-83e8-4470-9d3e-95f217a99a1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:00:44.350297 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:00:44.382801 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 825/1610] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:00:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 16:00:44.388125 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffae8abd-83e8-4470-9d3e-95f217a99a1c None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:00:44.393738 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:00:44.396276 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395b7351-509b-497b-a076-3792031642de admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:00:44.404234 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c227407-7116-4f58-b2f7-9e7bfd6796f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:00:44.406503 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c227407-7116-4f58-b2f7-9e7bfd6796f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:00:44.409373 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffae8abd-83e8-4470-9d3e-95f217a99a1c None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:00:44.409915 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c227407-7116-4f58-b2f7-9e7bfd6796f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:00:44.410330 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c227407-7116-4f58-b2f7-9e7bfd6796f7 admin admin] PATH_INFO: `/v3/projects/7412ae4990fa4fe782cc4c5e3db00d75/users/5dd4357556944c7089efb34ee58a8e04/roles/e2a5bbf3200a4d4091d72bc7ff78c34d` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:00:44.424667 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39a0896-be6d-4517-8d6a-6c7031df2176 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:00:44.425558 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39a0896-be6d-4517-8d6a-6c7031df2176 admin admin] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:00:44.425945 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39a0896-be6d-4517-8d6a-6c7031df2176 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:00:44.426510 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39a0896-be6d-4517-8d6a-6c7031df2176 admin admin] PATH_INFO: `/v3/users` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:00:44.427366 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39a0896-be6d-4517-8d6a-6c7031df2176 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:00:44.433904 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39a0896-be6d-4517-8d6a-6c7031df2176 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:00:44.442209 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c227407-7116-4f58-b2f7-9e7bfd6796f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7412ae4990fa4fe782cc4c5e3db00d75, user_id=5dd4357556944c7089efb34ee58a8e04, role_id=e2a5bbf3200a4d4091d72bc7ff78c34d)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:00:44.445912 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c227407-7116-4f58-b2f7-9e7bfd6796f7 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:00:44.477431 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 786/1611] 10.222.0.46 () {60 vars in 1482 bytes} [Thu Jun 23 16:00:44 2022] PUT /identity/v3/projects/7412ae4990fa4fe782cc4c5e3db00d75/users/5dd4357556944c7089efb34ee58a8e04/roles/e2a5bbf3200a4d4091d72bc7ff78c34d => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 16:00:44.481028 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8926f062-4390-442e-b092-f73d95f37e64 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:00:44.484695 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 826/1612] 10.222.0.46 () {60 vars in 1143 bytes} [Thu Jun 23 16:00:44 2022] POST /identity/v3/users => generated 308 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 16:00:44.484880 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8926f062-4390-442e-b092-f73d95f37e64 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:00:44.484880 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8926f062-4390-442e-b092-f73d95f37e64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:00:44.485176 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05972e8d-bf3e-49d5-9581-3a5086caf5fa None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:00:44.486092 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05972e8d-bf3e-49d5-9581-3a5086caf5fa None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:00:44.486475 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05972e8d-bf3e-49d5-9581-3a5086caf5fa None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:00:44.488417 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 827/1613] 10.222.0.46 () {54 vars in 961 bytes} [Thu Jun 23 16:00:44 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 16:00:44.494722 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd37c9e-156e-4e3c-a538-5ac95470cbb9 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:00:44.495330 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd37c9e-156e-4e3c-a538-5ac95470cbb9 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:00:44.496620 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd37c9e-156e-4e3c-a538-5ac95470cbb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:00:44.525134 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-8926f062-4390-442e-b092-f73d95f37e64 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:00:44.528621 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-8926f062-4390-442e-b092-f73d95f37e64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:00:44.569461 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-4fd37c9e-156e-4e3c-a538-5ac95470cbb9 None None] MFA Rules not processed for user `3263967dbd554d7bb70c8c0b074ff247`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:00:44.574014 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-4fd37c9e-156e-4e3c-a538-5ac95470cbb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:00:44.676705 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 787/1614] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:00:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 16:00:44.677478 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8926f062-4390-442e-b092-f73d95f37e64 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:00:44.736075 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 828/1615] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:00:44 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 16:00:44.738194 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fb66e6-a36f-447a-ae14-c7df4a89e410 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:00:44.738578 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fb66e6-a36f-447a-ae14-c7df4a89e410 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:00:44.739004 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fb66e6-a36f-447a-ae14-c7df4a89e410 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:00:44.769601 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-c6fb66e6-a36f-447a-ae14-c7df4a89e410 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:00:44.771170 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-c6fb66e6-a36f-447a-ae14-c7df4a89e410 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:00:44.781830 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8926f062-4390-442e-b092-f73d95f37e64 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:00:44.782155 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 829/1616] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:00:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 16:00:44.783333 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fb66e6-a36f-447a-ae14-c7df4a89e410 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:00:44.786650 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab1c94e-8594-470f-a03e-fc3e21d4502a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51784064951a4619ae09b9167fb0fd0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:00:44.787366 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab1c94e-8594-470f-a03e-fc3e21d4502a service neutron] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:00:44.787668 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab1c94e-8594-470f-a03e-fc3e21d4502a service neutron] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:00:44.788442 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab1c94e-8594-470f-a03e-fc3e21d4502a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:00:44.800490 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fb66e6-a36f-447a-ae14-c7df4a89e410 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:00:44.802517 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab1c94e-8594-470f-a03e-fc3e21d4502a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:00:44.804109 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab1c94e-8594-470f-a03e-fc3e21d4502a service neutron] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:00:44.815727 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5f3a09-c483-4131-bc88-690b484be6f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:00:44.817020 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 788/1617] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jun 23 16:00:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 16:00:44.817437 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5f3a09-c483-4131-bc88-690b484be6f7 admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:00:44.817903 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5f3a09-c483-4131-bc88-690b484be6f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:00:44.818363 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5f3a09-c483-4131-bc88-690b484be6f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:00:44.818848 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8b4e4e-838a-47ef-b26b-c6a1d9c5bbaf None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:00:44.819155 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8b4e4e-838a-47ef-b26b-c6a1d9c5bbaf None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:00:44.819440 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8b4e4e-838a-47ef-b26b-c6a1d9c5bbaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:00:44.820092 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5f3a09-c483-4131-bc88-690b484be6f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:00:44.824169 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5f3a09-c483-4131-bc88-690b484be6f7 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:00:44.843150 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 830/1618] 10.222.0.46 () {58 vars in 1121 bytes} [Thu Jun 23 16:00:44 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 16:00:44.844128 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5f3a09-c483-4131-bc88-690b484be6f7 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:00:44.845518 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-7c8b4e4e-838a-47ef-b26b-c6a1d9c5bbaf None None] MFA Rules not processed for user `705115d19bab424d894bea8b1b48a7f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:00:44.847178 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-7c8b4e4e-838a-47ef-b26b-c6a1d9c5bbaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:00:44.861204 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5f3a09-c483-4131-bc88-690b484be6f7 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:00:44.872681 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c741f17-6948-4e90-a9fc-5191422629e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51784064951a4619ae09b9167fb0fd0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:00:44.874681 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c741f17-6948-4e90-a9fc-5191422629e3 service neutron] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:00:44.875166 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c741f17-6948-4e90-a9fc-5191422629e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:00:44.875350 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c741f17-6948-4e90-a9fc-5191422629e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:00:44.887599 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c741f17-6948-4e90-a9fc-5191422629e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:00:44.889104 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c741f17-6948-4e90-a9fc-5191422629e3 service neutron] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:00:44.903592 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 831/1619] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jun 23 16:00:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 16:00:44.904730 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c741f17-6948-4e90-a9fc-5191422629e3 service neutron] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:00:44.917307 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c741f17-6948-4e90-a9fc-5191422629e3 service neutron] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:00:44.923315 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753cd9c7-b800-4ebe-a741-4fed95c78b11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:00:44.924228 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753cd9c7-b800-4ebe-a741-4fed95c78b11 admin admin] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:00:44.925385 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753cd9c7-b800-4ebe-a741-4fed95c78b11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:00:44.925605 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753cd9c7-b800-4ebe-a741-4fed95c78b11 admin admin] PATH_INFO: `/v3/projects` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:00:44.926679 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753cd9c7-b800-4ebe-a741-4fed95c78b11 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:00:44.930158 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753cd9c7-b800-4ebe-a741-4fed95c78b11 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:00:44.964621 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 789/1620] 10.222.0.46 () {58 vars in 1058 bytes} [Thu Jun 23 16:00:44 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 16:00:44.965591 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8b4e4e-838a-47ef-b26b-c6a1d9c5bbaf None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:00:44.971535 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:00:44.980663 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8b4e4e-838a-47ef-b26b-c6a1d9c5bbaf None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:00:44.991949 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 832/1621] 10.222.0.46 () {60 vars in 1152 bytes} [Thu Jun 23 16:00:44 2022] POST /identity/v3/projects => generated 379 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 16:00:44.993990 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753cd9c7-b800-4ebe-a741-4fed95c78b11 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:00:44.995170 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3cbeb8-b16b-4e8c-8f20-5fa04e5f3048 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51784064951a4619ae09b9167fb0fd0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:00:44.999219 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3cbeb8-b16b-4e8c-8f20-5fa04e5f3048 service neutron] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:00:44.999771 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3cbeb8-b16b-4e8c-8f20-5fa04e5f3048 service neutron] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:00:45.000435 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3cbeb8-b16b-4e8c-8f20-5fa04e5f3048 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:00:45.102051 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3cbeb8-b16b-4e8c-8f20-5fa04e5f3048 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:00:45.102051 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3cbeb8-b16b-4e8c-8f20-5fa04e5f3048 service neutron] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:00:45.102051 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 790/1622] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jun 23 16:00:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 16:00:45.102051 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3cbeb8-b16b-4e8c-8f20-5fa04e5f3048 service neutron] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:00:45.131426 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3cbeb8-b16b-4e8c-8f20-5fa04e5f3048 service neutron] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:00:45.168933 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753cd9c7-b800-4ebe-a741-4fed95c78b11 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:00:45.208115 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b2d889-af3a-44af-bb2e-1d10882ec2d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:00:45.209056 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b2d889-af3a-44af-bb2e-1d10882ec2d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:00:45.209327 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b2d889-af3a-44af-bb2e-1d10882ec2d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:00:45.209604 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b2d889-af3a-44af-bb2e-1d10882ec2d5 admin admin] PATH_INFO: `/v3/projects/4faff0e9bb0e497b96e4e9e19298b06f/users/29d0f25e94d34418a08e12d00fffc959/roles/812d3c567c52469cb2b35709fbdc857c` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:00:45.257436 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b2d889-af3a-44af-bb2e-1d10882ec2d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4faff0e9bb0e497b96e4e9e19298b06f, user_id=29d0f25e94d34418a08e12d00fffc959, role_id=812d3c567c52469cb2b35709fbdc857c)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:00:45.261051 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b2d889-af3a-44af-bb2e-1d10882ec2d5 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:00:45.298345 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 833/1623] 10.222.0.46 () {60 vars in 1482 bytes} [Thu Jun 23 16:00:44 2022] PUT /identity/v3/projects/4faff0e9bb0e497b96e4e9e19298b06f/users/29d0f25e94d34418a08e12d00fffc959/roles/812d3c567c52469cb2b35709fbdc857c => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 16:00:45.300252 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b2d889-af3a-44af-bb2e-1d10882ec2d5 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:00:45.308763 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fcf081-a54e-4930-9228-1a35af2c0117 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '705115d19bab424d894bea8b1b48a7f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:00:45.310424 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fcf081-a54e-4930-9228-1a35af2c0117 service nova] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:00:45.311241 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fcf081-a54e-4930-9228-1a35af2c0117 service nova] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:00:45.311241 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fcf081-a54e-4930-9228-1a35af2c0117 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:00:45.314474 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:00:45.440241 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fcf081-a54e-4930-9228-1a35af2c0117 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:00:45.442311 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fcf081-a54e-4930-9228-1a35af2c0117 service nova] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:00:45.472005 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 791/1624] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jun 23 16:00:45 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 16:00:45.473058 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fcf081-a54e-4930-9228-1a35af2c0117 service nova] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:00:45.481795 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b2d889-af3a-44af-bb2e-1d10882ec2d5 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:00:45.500600 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6717c18-d849-4f77-b6ca-ea596ce0005e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:00:45.503049 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6717c18-d849-4f77-b6ca-ea596ce0005e admin admin] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:00:45.504579 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6717c18-d849-4f77-b6ca-ea596ce0005e admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:00:45.505232 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fcf081-a54e-4930-9228-1a35af2c0117 service nova] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:00:45.505454 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6717c18-d849-4f77-b6ca-ea596ce0005e admin admin] PATH_INFO: `/v3/users` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:00:45.508934 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6717c18-d849-4f77-b6ca-ea596ce0005e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:00:45.511665 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a70811e-b722-4d7a-9b40-6484b1556498 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:00:45.512382 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a70811e-b722-4d7a-9b40-6484b1556498 admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:00:45.512668 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a70811e-b722-4d7a-9b40-6484b1556498 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:00:45.512949 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a70811e-b722-4d7a-9b40-6484b1556498 admin admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:00:45.513706 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a70811e-b722-4d7a-9b40-6484b1556498 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:00:45.519081 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a70811e-b722-4d7a-9b40-6484b1556498 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:00:45.519319 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6717c18-d849-4f77-b6ca-ea596ce0005e admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:00:45.547371 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 792/1625] 10.222.0.46 () {58 vars in 1121 bytes} [Thu Jun 23 16:00:45 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 16:00:45.553038 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a70811e-b722-4d7a-9b40-6484b1556498 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:00:45.583351 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 834/1626] 10.222.0.46 () {60 vars in 1143 bytes} [Thu Jun 23 16:00:45 2022] POST /identity/v3/users => generated 296 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 16:00:45.584558 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6717c18-d849-4f77-b6ca-ea596ce0005e admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:00:45.587314 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a70811e-b722-4d7a-9b40-6484b1556498 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:00:45.599144 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6717c18-d849-4f77-b6ca-ea596ce0005e admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:00:45.606101 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c3fb61-62c0-4b89-9bd0-36ae8af7e9c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:00:45.607088 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c3fb61-62c0-4b89-9bd0-36ae8af7e9c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:00:45.607379 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c3fb61-62c0-4b89-9bd0-36ae8af7e9c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:00:45.607736 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c3fb61-62c0-4b89-9bd0-36ae8af7e9c1 admin admin] PATH_INFO: `/v3/projects/4faff0e9bb0e497b96e4e9e19298b06f/users/29d0f25e94d34418a08e12d00fffc959/roles/812d3c567c52469cb2b35709fbdc857c` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:00:45.610896 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c3fb61-62c0-4b89-9bd0-36ae8af7e9c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4faff0e9bb0e497b96e4e9e19298b06f, user_id=29d0f25e94d34418a08e12d00fffc959, role_id=812d3c567c52469cb2b35709fbdc857c)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:00:45.613692 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c3fb61-62c0-4b89-9bd0-36ae8af7e9c1 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:00:45.621785 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4a4be9-e443-44d1-a497-4b9a58a04faa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:00:45.621785 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4a4be9-e443-44d1-a497-4b9a58a04faa admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:00:45.622504 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4a4be9-e443-44d1-a497-4b9a58a04faa admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:00:45.622664 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4a4be9-e443-44d1-a497-4b9a58a04faa admin admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:00:45.623983 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4a4be9-e443-44d1-a497-4b9a58a04faa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:00:45.628806 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4a4be9-e443-44d1-a497-4b9a58a04faa admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:00:45.660622 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 793/1627] 10.222.0.46 () {60 vars in 1482 bytes} [Thu Jun 23 16:00:45 2022] PUT /identity/v3/projects/4faff0e9bb0e497b96e4e9e19298b06f/users/29d0f25e94d34418a08e12d00fffc959/roles/812d3c567c52469cb2b35709fbdc857c => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 16:00:45.674318 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c3fb61-62c0-4b89-9bd0-36ae8af7e9c1 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:00:45.674993 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 835/1628] 10.222.0.46 () {58 vars in 1121 bytes} [Thu Jun 23 16:00:45 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 16:00:45.680493 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a3ba8a-f6ed-47dd-8035-99446b0620af None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:00:45.680865 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a3ba8a-f6ed-47dd-8035-99446b0620af None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:00:45.681338 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a3ba8a-f6ed-47dd-8035-99446b0620af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:00:45.777140 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-83a3ba8a-f6ed-47dd-8035-99446b0620af None None] MFA Rules not processed for user `29d0f25e94d34418a08e12d00fffc959`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:00:45.780647 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-83a3ba8a-f6ed-47dd-8035-99446b0620af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:00:45.859026 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c3fb61-62c0-4b89-9bd0-36ae8af7e9c1 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:00:45.875997 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d831e43d-2103-4286-88f7-c2a7edcd63c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:00:45.877372 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d831e43d-2103-4286-88f7-c2a7edcd63c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:00:45.877917 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d831e43d-2103-4286-88f7-c2a7edcd63c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:00:45.878544 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d831e43d-2103-4286-88f7-c2a7edcd63c6 admin admin] PATH_INFO: `/v3/projects/8ec6f914ff6d4ccda98d260d15c78e60/users/67ca75bef7a047e8966f7973a7e0d712/roles/812d3c567c52469cb2b35709fbdc857c` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:00:45.886551 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:00:45.944065 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d831e43d-2103-4286-88f7-c2a7edcd63c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ec6f914ff6d4ccda98d260d15c78e60, user_id=67ca75bef7a047e8966f7973a7e0d712, role_id=812d3c567c52469cb2b35709fbdc857c)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:00:45.950068 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d831e43d-2103-4286-88f7-c2a7edcd63c6 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:00:45.993525 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 794/1629] 10.222.0.46 () {60 vars in 1482 bytes} [Thu Jun 23 16:00:45 2022] PUT /identity/v3/projects/8ec6f914ff6d4ccda98d260d15c78e60/users/67ca75bef7a047e8966f7973a7e0d712/roles/812d3c567c52469cb2b35709fbdc857c => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 16:00:46.007091 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d831e43d-2103-4286-88f7-c2a7edcd63c6 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:00:46.056488 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 836/1630] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:00:45 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 377 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 16:00:46.074157 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a3ba8a-f6ed-47dd-8035-99446b0620af None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:00:46.221833 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a3ba8a-f6ed-47dd-8035-99446b0620af None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:00:46.227615 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d831e43d-2103-4286-88f7-c2a7edcd63c6 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:00:46.233389 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdf5434-4176-46a4-a399-3541a82e81c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:00:46.234500 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdf5434-4176-46a4-a399-3541a82e81c4 admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:00:46.234990 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdf5434-4176-46a4-a399-3541a82e81c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:00:46.235429 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdf5434-4176-46a4-a399-3541a82e81c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:00:46.238850 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bdf5434-4176-46a4-a399-3541a82e81c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:00:46.242609 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bdf5434-4176-46a4-a399-3541a82e81c4 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:00:46.256229 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 795/1631] 10.222.0.46 () {58 vars in 1121 bytes} [Thu Jun 23 16:00:46 2022] GET /identity/v3/roles => generated 1420 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 16:00:46.258519 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce651340-a6e9-4fd6-82a8-74b35784f9d3 None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:00:46.259071 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce651340-a6e9-4fd6-82a8-74b35784f9d3 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:00:46.259527 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce651340-a6e9-4fd6-82a8-74b35784f9d3 None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:00:46.266074 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:00:46.267818 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 796/1632] 10.222.0.46 () {54 vars in 912 bytes} [Thu Jun 23 16:00:46 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 16:00:46.269065 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce651340-a6e9-4fd6-82a8-74b35784f9d3 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:00:46.290730 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce651340-a6e9-4fd6-82a8-74b35784f9d3 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:00:46.301655 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01483c8c-caf3-4b62-a8b8-70fb9d1c84f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:00:46.302610 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01483c8c-caf3-4b62-a8b8-70fb9d1c84f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:00:46.302914 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01483c8c-caf3-4b62-a8b8-70fb9d1c84f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:00:46.303197 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01483c8c-caf3-4b62-a8b8-70fb9d1c84f8 admin admin] PATH_INFO: `/v3/projects/8ec6f914ff6d4ccda98d260d15c78e60/users/67ca75bef7a047e8966f7973a7e0d712/roles/812d3c567c52469cb2b35709fbdc857c` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:00:46.312152 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01483c8c-caf3-4b62-a8b8-70fb9d1c84f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ec6f914ff6d4ccda98d260d15c78e60, user_id=67ca75bef7a047e8966f7973a7e0d712, role_id=812d3c567c52469cb2b35709fbdc857c)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:00:46.315753 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01483c8c-caf3-4b62-a8b8-70fb9d1c84f8 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:00:46.338804 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 797/1633] 10.222.0.46 () {60 vars in 1482 bytes} [Thu Jun 23 16:00:46 2022] PUT /identity/v3/projects/8ec6f914ff6d4ccda98d260d15c78e60/users/67ca75bef7a047e8966f7973a7e0d712/roles/812d3c567c52469cb2b35709fbdc857c => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 16:00:46.340999 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469bf792-b6e2-467c-a837-42bbb09fd230 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce18eaaba3a6477684fff1fd98ec8b51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:00:46.341378 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785b9d09-b4c0-4118-bb1e-f3ff75f457a2 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:00:46.341697 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785b9d09-b4c0-4118-bb1e-f3ff75f457a2 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:00:46.341972 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785b9d09-b4c0-4118-bb1e-f3ff75f457a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:00:46.343827 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469bf792-b6e2-467c-a837-42bbb09fd230 service cinder] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:00:46.344113 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469bf792-b6e2-467c-a837-42bbb09fd230 service cinder] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:00:46.344388 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469bf792-b6e2-467c-a837-42bbb09fd230 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:00:46.386722 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-785b9d09-b4c0-4118-bb1e-f3ff75f457a2 None None] MFA Rules not processed for user `705115d19bab424d894bea8b1b48a7f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:00:46.387969 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-785b9d09-b4c0-4118-bb1e-f3ff75f457a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:00:46.493048 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469bf792-b6e2-467c-a837-42bbb09fd230 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:00:46.493048 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469bf792-b6e2-467c-a837-42bbb09fd230 service cinder] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:00:46.532594 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 837/1634] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jun 23 16:00:46 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 459 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 16:00:46.536343 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14eb3cf-5957-47b3-81a5-fe671a931e3a None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:00:46.536698 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14eb3cf-5957-47b3-81a5-fe671a931e3a None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:00:46.536982 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14eb3cf-5957-47b3-81a5-fe671a931e3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:00:46.605122 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-c14eb3cf-5957-47b3-81a5-fe671a931e3a None None] MFA Rules not processed for user `67ca75bef7a047e8966f7973a7e0d712`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:00:46.608576 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-c14eb3cf-5957-47b3-81a5-fe671a931e3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:00:46.652919 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 798/1635] 10.222.0.46 () {58 vars in 1009 bytes} [Thu Jun 23 16:00:46 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 315 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 16:00:46.668198 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785b9d09-b4c0-4118-bb1e-f3ff75f457a2 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:00:46.861820 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785b9d09-b4c0-4118-bb1e-f3ff75f457a2 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:00:46.879364 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 838/1636] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:00:46 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 346 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 16:00:46.879531 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb2a4af-e646-4b04-ac9a-cfd24a44c605 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce18eaaba3a6477684fff1fd98ec8b51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:00:46.880410 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb2a4af-e646-4b04-ac9a-cfd24a44c605 service cinder] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:00:46.880857 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb2a4af-e646-4b04-ac9a-cfd24a44c605 service cinder] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:00:46.881281 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb2a4af-e646-4b04-ac9a-cfd24a44c605 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:00:46.884722 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14eb3cf-5957-47b3-81a5-fe671a931e3a None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:00:46.908206 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb2a4af-e646-4b04-ac9a-cfd24a44c605 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:00:46.912462 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb2a4af-e646-4b04-ac9a-cfd24a44c605 service cinder] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:00:46.945401 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 799/1637] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jun 23 16:00:46 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 278 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 16:00:46.947783 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb2a4af-e646-4b04-ac9a-cfd24a44c605 service cinder] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:00:46.953448 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:00:46.970849 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb2a4af-e646-4b04-ac9a-cfd24a44c605 service cinder] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:00:46.983358 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ce3973-82c3-491b-b93d-719fa3b0937a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce18eaaba3a6477684fff1fd98ec8b51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:00:46.983952 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ce3973-82c3-491b-b93d-719fa3b0937a service cinder] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:00:46.984247 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ce3973-82c3-491b-b93d-719fa3b0937a service cinder] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:00:46.984591 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ce3973-82c3-491b-b93d-719fa3b0937a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:00:47.007527 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ce3973-82c3-491b-b93d-719fa3b0937a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:00:47.009540 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ce3973-82c3-491b-b93d-719fa3b0937a service cinder] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:00:47.031647 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 800/1638] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jun 23 16:00:46 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 16:00:47.193183 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14eb3cf-5957-47b3-81a5-fe671a931e3a None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:00:47.209920 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce23308-48ec-4b87-ba1d-b2723dfcf38f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:00:47.256428 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce23308-48ec-4b87-ba1d-b2723dfcf38f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:00:47.258808 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce23308-48ec-4b87-ba1d-b2723dfcf38f admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:00:47.263222 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce23308-48ec-4b87-ba1d-b2723dfcf38f admin admin] PATH_INFO: `/v3/users/743784269ad54fe295ada341b492739e` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:00:47.280707 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce23308-48ec-4b87-ba1d-b2723dfcf38f admin admin] RBAC: Authorizing `identity:delete_user(user_id=743784269ad54fe295ada341b492739e)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:00:47.289053 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce23308-48ec-4b87-ba1d-b2723dfcf38f admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:00:47.289887 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ce3973-82c3-491b-b93d-719fa3b0937a service cinder] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:00:47.295688 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:00:47.300113 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ce3973-82c3-491b-b93d-719fa3b0937a service cinder] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:00:47.321217 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fac10e-a286-4252-baa3-f59ace054ce2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce18eaaba3a6477684fff1fd98ec8b51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:00:47.322061 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fac10e-a286-4252-baa3-f59ace054ce2 service cinder] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:00:47.322376 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fac10e-a286-4252-baa3-f59ace054ce2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:00:47.322670 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fac10e-a286-4252-baa3-f59ace054ce2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:00:47.486978 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-fce23308-48ec-4b87-ba1d-b2723dfcf38f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '743784269ad54fe295ada341b492739e'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:00:47.488433 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fac10e-a286-4252-baa3-f59ace054ce2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:00:47.493877 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fac10e-a286-4252-baa3-f59ace054ce2 service cinder] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:00:47.505342 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-fce23308-48ec-4b87-ba1d-b2723dfcf38f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '743784269ad54fe295ada341b492739e'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:00:47.505551 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-fce23308-48ec-4b87-ba1d-b2723dfcf38f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '743784269ad54fe295ada341b492739e'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:00:47.506757 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-fce23308-48ec-4b87-ba1d-b2723dfcf38f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '743784269ad54fe295ada341b492739e'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:00:47.529094 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 801/1639] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jun 23 16:00:47 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 240 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 16:00:47.547005 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-fce23308-48ec-4b87-ba1d-b2723dfcf38f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '743784269ad54fe295ada341b492739e'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:00:47.586137 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 839/1640] 10.222.0.46 () {58 vars in 1223 bytes} [Thu Jun 23 16:00:46 2022] DELETE /identity/v3/users/743784269ad54fe295ada341b492739e => generated 0 bytes in 706 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 16:00:47.591401 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2155ebf-b264-45b2-8166-e09bec8e7589 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:00:47.591710 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2155ebf-b264-45b2-8166-e09bec8e7589 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:00:47.591973 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2155ebf-b264-45b2-8166-e09bec8e7589 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:00:47.643083 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-c2155ebf-b264-45b2-8166-e09bec8e7589 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:00:47.644500 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-c2155ebf-b264-45b2-8166-e09bec8e7589 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:00:47.849778 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 802/1641] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:00:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 16:00:47.986099 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce23308-48ec-4b87-ba1d-b2723dfcf38f admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:00:47.991590 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-fce23308-48ec-4b87-ba1d-b2723dfcf38f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:00:48.169563 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce23308-48ec-4b87-ba1d-b2723dfcf38f admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:00:48.182720 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80922454-380a-44a7-aeea-03198224e40d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51784064951a4619ae09b9167fb0fd0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:00:48.185033 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80922454-380a-44a7-aeea-03198224e40d service neutron] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:00:48.186776 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80922454-380a-44a7-aeea-03198224e40d service neutron] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:00:48.187187 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80922454-380a-44a7-aeea-03198224e40d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:00:48.204738 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80922454-380a-44a7-aeea-03198224e40d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:00:48.206589 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80922454-380a-44a7-aeea-03198224e40d service neutron] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:00:48.228715 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 840/1642] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jun 23 16:00:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 243 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 16:00:48.415635 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca94eb9-46fa-46e9-b70f-2114b7072068 None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:00:48.417335 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca94eb9-46fa-46e9-b70f-2114b7072068 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:00:48.417863 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca94eb9-46fa-46e9-b70f-2114b7072068 None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:00:48.420147 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 803/1643] 10.222.0.46 () {54 vars in 961 bytes} [Thu Jun 23 16:00:48 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 16:00:48.432722 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4443d197-db6b-4111-91e8-ca3f5eacc935 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:00:48.433961 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4443d197-db6b-4111-91e8-ca3f5eacc935 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:00:48.434632 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4443d197-db6b-4111-91e8-ca3f5eacc935 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:00:48.479918 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-4443d197-db6b-4111-91e8-ca3f5eacc935 None None] MFA Rules not processed for user `705115d19bab424d894bea8b1b48a7f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:00:48.481328 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-4443d197-db6b-4111-91e8-ca3f5eacc935 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:00:48.806206 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 841/1644] 10.222.0.46 () {58 vars in 1058 bytes} [Thu Jun 23 16:00:48 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 375 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 16:00:48.827999 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90a1be5-26e4-4614-98be-28834deae158 None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:00:48.829023 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90a1be5-26e4-4614-98be-28834deae158 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:00:48.829424 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90a1be5-26e4-4614-98be-28834deae158 None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:00:48.831080 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 804/1645] 10.222.0.46 () {54 vars in 912 bytes} [Thu Jun 23 16:00:48 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 16:00:48.868041 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf01392e-a1ed-42d3-8ed6-89af0f81bb8c None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:00:48.868520 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf01392e-a1ed-42d3-8ed6-89af0f81bb8c None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:00:48.868801 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf01392e-a1ed-42d3-8ed6-89af0f81bb8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:00:48.913462 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-cf01392e-a1ed-42d3-8ed6-89af0f81bb8c None None] MFA Rules not processed for user `705115d19bab424d894bea8b1b48a7f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:00:48.914744 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-cf01392e-a1ed-42d3-8ed6-89af0f81bb8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:00:48.933243 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 842/1646] 10.222.0.46 () {58 vars in 1009 bytes} [Thu Jun 23 16:00:48 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 16:00:48.975063 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90a1be5-26e4-4614-98be-28834deae158 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:00:48.977001 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-e90a1be5-26e4-4614-98be-28834deae158 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:00:49.094398 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90a1be5-26e4-4614-98be-28834deae158 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:00:49.105671 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffa6de6-df90-4e3d-a236-ea7c0cdeffd9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce18eaaba3a6477684fff1fd98ec8b51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:00:49.108644 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffa6de6-df90-4e3d-a236-ea7c0cdeffd9 service cinder] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:00:49.108940 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffa6de6-df90-4e3d-a236-ea7c0cdeffd9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:00:49.109224 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffa6de6-df90-4e3d-a236-ea7c0cdeffd9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:00:49.129028 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffa6de6-df90-4e3d-a236-ea7c0cdeffd9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:00:49.131614 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffa6de6-df90-4e3d-a236-ea7c0cdeffd9 service cinder] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:00:49.217600 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 805/1647] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jun 23 16:00:48 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 243 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 16:00:50.654934 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf01392e-a1ed-42d3-8ed6-89af0f81bb8c None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:00:50.656419 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-cf01392e-a1ed-42d3-8ed6-89af0f81bb8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:00:50.679504 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf01392e-a1ed-42d3-8ed6-89af0f81bb8c None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:00:50.700180 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642cf5a1-0d79-49fe-adb5-aee878041162 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:00:50.701150 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642cf5a1-0d79-49fe-adb5-aee878041162 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:00:50.701509 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642cf5a1-0d79-49fe-adb5-aee878041162 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:00:50.701829 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642cf5a1-0d79-49fe-adb5-aee878041162 admin admin] PATH_INFO: `/v3/projects/d91ab0ef24e84c07ad4948303d1f184c` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:00:50.705516 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642cf5a1-0d79-49fe-adb5-aee878041162 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d91ab0ef24e84c07ad4948303d1f184c)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:00:50.715733 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: WARNING py.warnings [None req-642cf5a1-0d79-49fe-adb5-aee878041162 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 23 16:00:50.715733 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: warnings.warn(msg) Jun 23 16:00:50.715733 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: Jun 23 16:00:50.716975 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642cf5a1-0d79-49fe-adb5-aee878041162 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:00:50.719150 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:00:50.906135 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-642cf5a1-0d79-49fe-adb5-aee878041162 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd91ab0ef24e84c07ad4948303d1f184c', 'request_id': 'req-642cf5a1-0d79-49fe-adb5-aee878041162'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:00:50.933402 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-642cf5a1-0d79-49fe-adb5-aee878041162 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd91ab0ef24e84c07ad4948303d1f184c', 'request_id': 'req-642cf5a1-0d79-49fe-adb5-aee878041162'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:00:50.976437 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-642cf5a1-0d79-49fe-adb5-aee878041162 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd91ab0ef24e84c07ad4948303d1f184c', 'request_id': 'req-642cf5a1-0d79-49fe-adb5-aee878041162'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:00:51.013339 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-642cf5a1-0d79-49fe-adb5-aee878041162 admin admin] The token cache is being invalidate because project d91ab0ef24e84c07ad4948303d1f184c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80153) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 16:00:51.013784 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-642cf5a1-0d79-49fe-adb5-aee878041162 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:00:51.015200 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-642cf5a1-0d79-49fe-adb5-aee878041162 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:00:51.017319 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 843/1648] 10.222.0.46 () {58 vars in 1232 bytes} [Thu Jun 23 16:00:50 2022] DELETE /identity/v3/projects/d91ab0ef24e84c07ad4948303d1f184c => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 16:00:51.022117 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffa6de6-df90-4e3d-a236-ea7c0cdeffd9 service cinder] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:00:51.038952 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffa6de6-df90-4e3d-a236-ea7c0cdeffd9 service cinder] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:00:51.136799 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:00:51.209649 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ab3b48-710b-4ea2-b915-1c167f496686 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:00:51.210617 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ab3b48-710b-4ea2-b915-1c167f496686 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:00:51.210971 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ab3b48-710b-4ea2-b915-1c167f496686 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:00:51.211427 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ab3b48-710b-4ea2-b915-1c167f496686 admin admin] PATH_INFO: `/v3/users/29d0f25e94d34418a08e12d00fffc959` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:00:51.213777 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ab3b48-710b-4ea2-b915-1c167f496686 admin admin] RBAC: Authorizing `identity:delete_user(user_id=29d0f25e94d34418a08e12d00fffc959)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:00:51.215597 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: WARNING py.warnings [None req-c2ab3b48-710b-4ea2-b915-1c167f496686 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 23 16:00:51.215597 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: warnings.warn(msg) Jun 23 16:00:51.215597 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: Jun 23 16:00:51.216852 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ab3b48-710b-4ea2-b915-1c167f496686 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:00:51.356598 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-c2ab3b48-710b-4ea2-b915-1c167f496686 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '29d0f25e94d34418a08e12d00fffc959'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:00:51.357310 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-c2ab3b48-710b-4ea2-b915-1c167f496686 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '29d0f25e94d34418a08e12d00fffc959'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:00:51.377909 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-c2ab3b48-710b-4ea2-b915-1c167f496686 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '29d0f25e94d34418a08e12d00fffc959'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:00:51.397290 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-c2ab3b48-710b-4ea2-b915-1c167f496686 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '29d0f25e94d34418a08e12d00fffc959'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:00:51.397932 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-c2ab3b48-710b-4ea2-b915-1c167f496686 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '29d0f25e94d34418a08e12d00fffc959'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:00:51.453665 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 806/1649] 10.222.0.46 () {58 vars in 1223 bytes} [Thu Jun 23 16:00:51 2022] DELETE /identity/v3/users/29d0f25e94d34418a08e12d00fffc959 => generated 0 bytes in 432 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 16:00:51.655155 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbca8a17-22d6-462a-afd2-ad4178f3394b None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:00:51.657178 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbca8a17-22d6-462a-afd2-ad4178f3394b None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:00:51.657727 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbca8a17-22d6-462a-afd2-ad4178f3394b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:00:51.735677 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-dbca8a17-22d6-462a-afd2-ad4178f3394b None None] MFA Rules not processed for user `82e02e90305a424c8e719a92bd084a5f`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:00:51.738466 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-dbca8a17-22d6-462a-afd2-ad4178f3394b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:00:51.916061 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 844/1650] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:00:51 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 16:00:51.928260 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2578f261-ae8a-4e23-9ae4-165a34c4c5b3 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:00:51.928648 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2578f261-ae8a-4e23-9ae4-165a34c4c5b3 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:00:51.928938 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2578f261-ae8a-4e23-9ae4-165a34c4c5b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:00:51.972661 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-2578f261-ae8a-4e23-9ae4-165a34c4c5b3 None None] MFA Rules not processed for user `82e02e90305a424c8e719a92bd084a5f`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:00:51.977050 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-2578f261-ae8a-4e23-9ae4-165a34c4c5b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:00:51.991828 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 807/1651] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:00:51 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 16:00:52.007711 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbca8a17-22d6-462a-afd2-ad4178f3394b None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:00:52.010007 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-dbca8a17-22d6-462a-afd2-ad4178f3394b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:00:52.144353 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbca8a17-22d6-462a-afd2-ad4178f3394b None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:00:52.164984 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc39ecc-e7b4-4dca-9ac4-840f184a645f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '705115d19bab424d894bea8b1b48a7f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:00:52.165746 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc39ecc-e7b4-4dca-9ac4-840f184a645f service nova] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:00:52.166044 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc39ecc-e7b4-4dca-9ac4-840f184a645f service nova] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:00:52.166359 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc39ecc-e7b4-4dca-9ac4-840f184a645f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:00:52.187418 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc39ecc-e7b4-4dca-9ac4-840f184a645f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:00:52.189451 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc39ecc-e7b4-4dca-9ac4-840f184a645f service nova] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:00:52.214781 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 845/1652] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jun 23 16:00:52 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 16:00:52.681613 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2578f261-ae8a-4e23-9ae4-165a34c4c5b3 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:00:52.683683 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-2578f261-ae8a-4e23-9ae4-165a34c4c5b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:00:52.788011 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2578f261-ae8a-4e23-9ae4-165a34c4c5b3 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:00:52.815916 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c35179-5074-4680-9ffc-7b37d7dbd79d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:00:52.819498 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c35179-5074-4680-9ffc-7b37d7dbd79d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:00:52.819803 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c35179-5074-4680-9ffc-7b37d7dbd79d admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:00:52.820091 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c35179-5074-4680-9ffc-7b37d7dbd79d admin admin] PATH_INFO: `/v3/projects/4faff0e9bb0e497b96e4e9e19298b06f` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:00:52.836260 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c35179-5074-4680-9ffc-7b37d7dbd79d admin admin] RBAC: Authorizing `identity:delete_project(project_id=4faff0e9bb0e497b96e4e9e19298b06f)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:00:52.839796 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c35179-5074-4680-9ffc-7b37d7dbd79d admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:00:52.976913 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:00:52.988217 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-d6c35179-5074-4680-9ffc-7b37d7dbd79d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4faff0e9bb0e497b96e4e9e19298b06f', 'request_id': 'req-d6c35179-5074-4680-9ffc-7b37d7dbd79d'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:00:52.998893 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-d6c35179-5074-4680-9ffc-7b37d7dbd79d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4faff0e9bb0e497b96e4e9e19298b06f', 'request_id': 'req-d6c35179-5074-4680-9ffc-7b37d7dbd79d'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:00:53.008616 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-d6c35179-5074-4680-9ffc-7b37d7dbd79d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4faff0e9bb0e497b96e4e9e19298b06f', 'request_id': 'req-d6c35179-5074-4680-9ffc-7b37d7dbd79d'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:00:53.024841 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-d6c35179-5074-4680-9ffc-7b37d7dbd79d admin admin] The token cache is being invalidate because project 4faff0e9bb0e497b96e4e9e19298b06f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80154) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 16:00:53.027289 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-d6c35179-5074-4680-9ffc-7b37d7dbd79d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:00:53.028213 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-d6c35179-5074-4680-9ffc-7b37d7dbd79d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:00:53.035432 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 808/1653] 10.222.0.46 () {58 vars in 1232 bytes} [Thu Jun 23 16:00:52 2022] DELETE /identity/v3/projects/4faff0e9bb0e497b96e4e9e19298b06f => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 16:00:53.037235 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9e18b7-ee3a-44b2-809c-48771c24d7c7 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:00:53.037702 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9e18b7-ee3a-44b2-809c-48771c24d7c7 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:00:53.037783 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9e18b7-ee3a-44b2-809c-48771c24d7c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:00:53.047009 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:00:53.060470 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-af9e18b7-ee3a-44b2-809c-48771c24d7c7 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:00:53.063599 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a07c701-3244-417a-a7a2-a4277d6d447a None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:00:53.064093 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a07c701-3244-417a-a7a2-a4277d6d447a None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:00:53.064511 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a07c701-3244-417a-a7a2-a4277d6d447a None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:00:53.068029 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 809/1654] 10.222.0.46 () {54 vars in 912 bytes} [Thu Jun 23 16:00:53 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 16:00:53.073684 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61ce170-559b-4c68-984c-5ff5f2be644e None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:00:53.074198 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61ce170-559b-4c68-984c-5ff5f2be644e None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:00:53.074620 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61ce170-559b-4c68-984c-5ff5f2be644e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:00:53.079525 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-af9e18b7-ee3a-44b2-809c-48771c24d7c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:00:53.156890 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-e61ce170-559b-4c68-984c-5ff5f2be644e None None] MFA Rules not processed for user `51784064951a4619ae09b9167fb0fd0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:00:53.159389 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-e61ce170-559b-4c68-984c-5ff5f2be644e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:00:53.251018 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 846/1655] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:00:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 16:00:53.256958 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbf1060-b77c-4f16-bcd6-4a06d2fcdc2b None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:00:53.264187 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbf1060-b77c-4f16-bcd6-4a06d2fcdc2b None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:00:53.264642 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbf1060-b77c-4f16-bcd6-4a06d2fcdc2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:00:53.303438 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-0cbf1060-b77c-4f16-bcd6-4a06d2fcdc2b None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:00:53.304798 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-0cbf1060-b77c-4f16-bcd6-4a06d2fcdc2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:00:53.308591 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 810/1656] 10.222.0.46 () {58 vars in 1009 bytes} [Thu Jun 23 16:00:53 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 16:00:53.332980 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 847/1657] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:00:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 16:00:53.337151 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61ce170-559b-4c68-984c-5ff5f2be644e None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:00:53.353710 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61ce170-559b-4c68-984c-5ff5f2be644e None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:00:53.390334 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0090ff-1c4a-43db-a664-4e53a92afc1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:00:53.391513 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0090ff-1c4a-43db-a664-4e53a92afc1d admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:00:53.391945 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0090ff-1c4a-43db-a664-4e53a92afc1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:00:53.392494 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0090ff-1c4a-43db-a664-4e53a92afc1d admin admin] PATH_INFO: `/v3/domains` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:00:53.393400 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0090ff-1c4a-43db-a664-4e53a92afc1d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 16:00:53.393850 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0090ff-1c4a-43db-a664-4e53a92afc1d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:00:53.395866 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0090ff-1c4a-43db-a664-4e53a92afc1d admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:00:53.424598 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 811/1658] 10.222.0.46 () {58 vars in 1152 bytes} [Thu Jun 23 16:00:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 16:00:53.428004 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbf1060-b77c-4f16-bcd6-4a06d2fcdc2b None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:00:53.429490 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-0cbf1060-b77c-4f16-bcd6-4a06d2fcdc2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:00:53.431543 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0090ff-1c4a-43db-a664-4e53a92afc1d admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:00:53.459059 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbf1060-b77c-4f16-bcd6-4a06d2fcdc2b None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:00:53.465268 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0090ff-1c4a-43db-a664-4e53a92afc1d admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:00:53.475030 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab41f19-f8a5-462f-83f7-a3c005bdd0bb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51784064951a4619ae09b9167fb0fd0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:00:53.475720 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab41f19-f8a5-462f-83f7-a3c005bdd0bb service neutron] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:00:53.476008 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab41f19-f8a5-462f-83f7-a3c005bdd0bb service neutron] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:00:53.478079 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab41f19-f8a5-462f-83f7-a3c005bdd0bb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:00:53.487384 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4216175-390a-4c93-8edd-7f6bf8ef1d73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:00:53.488237 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4216175-390a-4c93-8edd-7f6bf8ef1d73 admin admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:00:53.488622 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4216175-390a-4c93-8edd-7f6bf8ef1d73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:00:53.492421 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4216175-390a-4c93-8edd-7f6bf8ef1d73 admin admin] PATH_INFO: `/v3/projects` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:00:53.493591 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4216175-390a-4c93-8edd-7f6bf8ef1d73 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:00:53.500726 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4216175-390a-4c93-8edd-7f6bf8ef1d73 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:00:53.505756 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab41f19-f8a5-462f-83f7-a3c005bdd0bb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:00:53.509860 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab41f19-f8a5-462f-83f7-a3c005bdd0bb service neutron] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:00:53.531752 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 848/1659] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jun 23 16:00:53 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 16:00:53.559274 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 812/1660] 10.222.0.46 () {60 vars in 1152 bytes} [Thu Jun 23 16:00:53 2022] POST /identity/v3/projects => generated 371 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 16:00:53.563613 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab41f19-f8a5-462f-83f7-a3c005bdd0bb service neutron] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:00:53.585080 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab41f19-f8a5-462f-83f7-a3c005bdd0bb service neutron] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:00:53.754333 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789ebc7a-cbf8-407a-a5b0-5c2aeb3abaa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:00:53.756090 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789ebc7a-cbf8-407a-a5b0-5c2aeb3abaa6 admin admin] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:00:53.756090 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789ebc7a-cbf8-407a-a5b0-5c2aeb3abaa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:00:53.756090 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789ebc7a-cbf8-407a-a5b0-5c2aeb3abaa6 admin admin] PATH_INFO: `/v3/users` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:00:53.756786 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789ebc7a-cbf8-407a-a5b0-5c2aeb3abaa6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:00:53.759992 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789ebc7a-cbf8-407a-a5b0-5c2aeb3abaa6 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:00:53.795776 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 849/1661] 10.222.0.46 () {60 vars in 1143 bytes} [Thu Jun 23 16:00:53 2022] POST /identity/v3/users => generated 286 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 16:00:53.801109 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4216175-390a-4c93-8edd-7f6bf8ef1d73 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:00:53.816087 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4216175-390a-4c93-8edd-7f6bf8ef1d73 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:00:53.831044 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e267e6d-a83b-4c56-9a9b-088b97583fdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:00:53.831919 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e267e6d-a83b-4c56-9a9b-088b97583fdc admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:00:53.832317 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e267e6d-a83b-4c56-9a9b-088b97583fdc admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:00:53.832726 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e267e6d-a83b-4c56-9a9b-088b97583fdc admin admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:00:53.833612 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e267e6d-a83b-4c56-9a9b-088b97583fdc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:00:53.836340 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e267e6d-a83b-4c56-9a9b-088b97583fdc admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:00:53.862973 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789ebc7a-cbf8-407a-a5b0-5c2aeb3abaa6 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:00:53.863761 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 813/1662] 10.222.0.46 () {58 vars in 1121 bytes} [Thu Jun 23 16:00:53 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 16:00:53.878354 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789ebc7a-cbf8-407a-a5b0-5c2aeb3abaa6 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:00:53.891499 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f15d1a-5949-4528-b0f7-a60027ee549a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:00:53.895342 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f15d1a-5949-4528-b0f7-a60027ee549a admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:00:53.896125 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f15d1a-5949-4528-b0f7-a60027ee549a admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:00:53.897852 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f15d1a-5949-4528-b0f7-a60027ee549a admin admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:00:53.899900 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f15d1a-5949-4528-b0f7-a60027ee549a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:00:53.903995 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f15d1a-5949-4528-b0f7-a60027ee549a admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:00:53.937705 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 850/1663] 10.222.0.46 () {58 vars in 1121 bytes} [Thu Jun 23 16:00:53 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 16:00:53.942554 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e267e6d-a83b-4c56-9a9b-088b97583fdc admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:00:53.958822 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e267e6d-a83b-4c56-9a9b-088b97583fdc admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:00:53.973878 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e263333-cea9-4fce-bd93-a7927259394f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:00:53.975030 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e263333-cea9-4fce-bd93-a7927259394f admin admin] REQUEST_METHOD: `PUT` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:00:53.975395 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e263333-cea9-4fce-bd93-a7927259394f admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:00:53.975756 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e263333-cea9-4fce-bd93-a7927259394f admin admin] PATH_INFO: `/v3/projects/6f6106af202449118ab58cfb0197236b/users/ad0a2a0ec6644b90bc67a9a47b95d7be/roles/e2a5bbf3200a4d4091d72bc7ff78c34d` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:00:54.013291 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e263333-cea9-4fce-bd93-a7927259394f admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f6106af202449118ab58cfb0197236b, user_id=ad0a2a0ec6644b90bc67a9a47b95d7be, role_id=e2a5bbf3200a4d4091d72bc7ff78c34d)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:00:54.020776 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e263333-cea9-4fce-bd93-a7927259394f admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:00:54.053739 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 814/1664] 10.222.0.46 () {60 vars in 1482 bytes} [Thu Jun 23 16:00:53 2022] PUT /identity/v3/projects/6f6106af202449118ab58cfb0197236b/users/ad0a2a0ec6644b90bc67a9a47b95d7be/roles/e2a5bbf3200a4d4091d72bc7ff78c34d => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 16:00:54.067363 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28811beb-20e0-4c61-b6a3-4e313ef58e8e None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:00:54.067785 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28811beb-20e0-4c61-b6a3-4e313ef58e8e None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:00:54.068278 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28811beb-20e0-4c61-b6a3-4e313ef58e8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:00:54.124398 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-28811beb-20e0-4c61-b6a3-4e313ef58e8e None None] MFA Rules not processed for user `ad0a2a0ec6644b90bc67a9a47b95d7be`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:00:54.130191 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-28811beb-20e0-4c61-b6a3-4e313ef58e8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:00:54.236811 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:00:54.278770 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 851/1665] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:00:54 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 16:00:54.283596 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e263333-cea9-4fce-bd93-a7927259394f admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:00:54.424918 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e263333-cea9-4fce-bd93-a7927259394f admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:00:54.462494 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c3835c-795a-41cc-a7f9-02bb68deab1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:00:54.465781 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c3835c-795a-41cc-a7f9-02bb68deab1f admin admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:00:54.469920 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c3835c-795a-41cc-a7f9-02bb68deab1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:00:54.470218 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c3835c-795a-41cc-a7f9-02bb68deab1f admin admin] PATH_INFO: `/v3/projects` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:00:54.471021 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c3835c-795a-41cc-a7f9-02bb68deab1f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:00:54.473057 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c3835c-795a-41cc-a7f9-02bb68deab1f admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:00:54.513795 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:DELETE=4,keystone:SELECT=93,keystone:INSERT=3 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:00:54.584113 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 815/1666] 10.222.0.46 () {60 vars in 1152 bytes} [Thu Jun 23 16:00:54 2022] POST /identity/v3/projects => generated 369 bytes in 301 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 16:00:54.591664 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28811beb-20e0-4c61-b6a3-4e313ef58e8e None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:00:54.622047 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28811beb-20e0-4c61-b6a3-4e313ef58e8e None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:00:54.801085 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86058a6b-1076-45b9-afa3-d56fa7fff94f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:00:54.801795 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86058a6b-1076-45b9-afa3-d56fa7fff94f admin admin] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:00:54.802761 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86058a6b-1076-45b9-afa3-d56fa7fff94f admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:00:54.803054 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86058a6b-1076-45b9-afa3-d56fa7fff94f admin admin] PATH_INFO: `/v3/users` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:00:54.803795 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86058a6b-1076-45b9-afa3-d56fa7fff94f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:00:54.805836 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86058a6b-1076-45b9-afa3-d56fa7fff94f admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:00:54.851449 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 852/1667] 10.222.0.46 () {60 vars in 1143 bytes} [Thu Jun 23 16:00:54 2022] POST /identity/v3/users => generated 291 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 16:00:54.855442 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c3835c-795a-41cc-a7f9-02bb68deab1f admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:00:54.874529 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c3835c-795a-41cc-a7f9-02bb68deab1f admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:00:54.886536 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9e2d80-bc5e-4593-a077-cc05d983f453 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:00:54.887912 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9e2d80-bc5e-4593-a077-cc05d983f453 admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:00:54.888404 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9e2d80-bc5e-4593-a077-cc05d983f453 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:00:54.888954 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9e2d80-bc5e-4593-a077-cc05d983f453 admin admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:00:54.889971 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9e2d80-bc5e-4593-a077-cc05d983f453 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:00:54.892736 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9e2d80-bc5e-4593-a077-cc05d983f453 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:00:54.925192 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86058a6b-1076-45b9-afa3-d56fa7fff94f admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:00:54.926150 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 816/1668] 10.222.0.46 () {58 vars in 1121 bytes} [Thu Jun 23 16:00:54 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 16:00:54.942383 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86058a6b-1076-45b9-afa3-d56fa7fff94f admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:00:54.953135 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c691a8ed-3223-4dc3-8e44-515fcc0ea174 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:00:54.955575 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c691a8ed-3223-4dc3-8e44-515fcc0ea174 admin admin] REQUEST_METHOD: `PUT` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:00:54.967845 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c691a8ed-3223-4dc3-8e44-515fcc0ea174 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:00:54.968415 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c691a8ed-3223-4dc3-8e44-515fcc0ea174 admin admin] PATH_INFO: `/v3/projects/03c8ea51d75b4c22a676dd24bc046195/users/acf8ceb77ac14478a95db91927a5f699/roles/812d3c567c52469cb2b35709fbdc857c` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:00:55.000383 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c691a8ed-3223-4dc3-8e44-515fcc0ea174 admin admin] RBAC: Authorizing `identity:create_grant(project_id=03c8ea51d75b4c22a676dd24bc046195, user_id=acf8ceb77ac14478a95db91927a5f699, role_id=812d3c567c52469cb2b35709fbdc857c)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:00:55.004107 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c691a8ed-3223-4dc3-8e44-515fcc0ea174 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:00:55.049407 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 853/1669] 10.222.0.46 () {60 vars in 1482 bytes} [Thu Jun 23 16:00:54 2022] PUT /identity/v3/projects/03c8ea51d75b4c22a676dd24bc046195/users/acf8ceb77ac14478a95db91927a5f699/roles/812d3c567c52469cb2b35709fbdc857c => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 16:00:55.050728 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9e2d80-bc5e-4593-a077-cc05d983f453 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:00:55.067091 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9e2d80-bc5e-4593-a077-cc05d983f453 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:00:55.303467 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f008c955-9e55-41ad-ba2c-e742a91bacd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:00:55.304203 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f008c955-9e55-41ad-ba2c-e742a91bacd7 admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:00:55.304471 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f008c955-9e55-41ad-ba2c-e742a91bacd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:00:55.304916 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f008c955-9e55-41ad-ba2c-e742a91bacd7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:00:55.305701 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f008c955-9e55-41ad-ba2c-e742a91bacd7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:00:55.307442 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f008c955-9e55-41ad-ba2c-e742a91bacd7 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:00:55.320087 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0291c5b9-7bae-4aaf-9a40-e8d7931ef00b None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:00:55.320607 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0291c5b9-7bae-4aaf-9a40-e8d7931ef00b None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:00:55.320691 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0291c5b9-7bae-4aaf-9a40-e8d7931ef00b None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:00:55.322927 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 854/1670] 10.222.0.46 () {54 vars in 960 bytes} [Thu Jun 23 16:00:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 16:00:55.335849 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43387d63-951f-4311-904f-630e954d537e None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:00:55.336135 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43387d63-951f-4311-904f-630e954d537e None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:00:55.336424 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43387d63-951f-4311-904f-630e954d537e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:00:55.387583 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 817/1671] 10.222.0.46 () {58 vars in 1121 bytes} [Thu Jun 23 16:00:55 2022] GET /identity/v3/roles => generated 1420 bytes in 337 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 16:00:55.392649 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f008c955-9e55-41ad-ba2c-e742a91bacd7 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:00:55.410789 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-43387d63-951f-4311-904f-630e954d537e None None] MFA Rules not processed for user `3f66988b73644fe38b9849663ee70f25`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:00:55.412512 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-43387d63-951f-4311-904f-630e954d537e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:00:55.445348 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f008c955-9e55-41ad-ba2c-e742a91bacd7 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:00:55.461193 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1823118-4597-4c05-b095-7ff1e954a213 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:00:55.464466 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1823118-4597-4c05-b095-7ff1e954a213 admin admin] REQUEST_METHOD: `PUT` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:00:55.467817 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1823118-4597-4c05-b095-7ff1e954a213 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:00:55.468308 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1823118-4597-4c05-b095-7ff1e954a213 admin admin] PATH_INFO: `/v3/projects/03c8ea51d75b4c22a676dd24bc046195/users/acf8ceb77ac14478a95db91927a5f699/roles/812d3c567c52469cb2b35709fbdc857c` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:00:55.471782 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1823118-4597-4c05-b095-7ff1e954a213 admin admin] RBAC: Authorizing `identity:create_grant(project_id=03c8ea51d75b4c22a676dd24bc046195, user_id=acf8ceb77ac14478a95db91927a5f699, role_id=812d3c567c52469cb2b35709fbdc857c)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:00:55.478103 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1823118-4597-4c05-b095-7ff1e954a213 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:00:55.501547 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 818/1672] 10.222.0.46 () {60 vars in 1482 bytes} [Thu Jun 23 16:00:55 2022] PUT /identity/v3/projects/03c8ea51d75b4c22a676dd24bc046195/users/acf8ceb77ac14478a95db91927a5f699/roles/812d3c567c52469cb2b35709fbdc857c => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 16:00:55.526928 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845ed9a0-6aa0-43a7-be69-2cb422ac2c54 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:00:55.541291 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845ed9a0-6aa0-43a7-be69-2cb422ac2c54 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:00:55.541990 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845ed9a0-6aa0-43a7-be69-2cb422ac2c54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:00:55.705561 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-845ed9a0-6aa0-43a7-be69-2cb422ac2c54 None None] MFA Rules not processed for user `acf8ceb77ac14478a95db91927a5f699`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:00:55.709946 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-845ed9a0-6aa0-43a7-be69-2cb422ac2c54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:00:55.751076 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 855/1673] 10.222.0.46 () {58 vars in 1057 bytes} [Thu Jun 23 16:00:55 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 417 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) Jun 23 16:00:55.761220 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43387d63-951f-4311-904f-630e954d537e None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:00:55.794429 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43387d63-951f-4311-904f-630e954d537e None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:00:56.098714 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 819/1674] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:00:55 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 586 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 16:00:56.100623 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:00:56.120708 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845ed9a0-6aa0-43a7-be69-2cb422ac2c54 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:00:56.122654 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-845ed9a0-6aa0-43a7-be69-2cb422ac2c54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:00:56.142623 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fbd59a-d541-4dfa-adaf-288ec9d17e23 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f66988b73644fe38b9849663ee70f25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:00:56.143516 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fbd59a-d541-4dfa-adaf-288ec9d17e23 service placement] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:00:56.143908 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fbd59a-d541-4dfa-adaf-288ec9d17e23 service placement] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:00:56.144325 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fbd59a-d541-4dfa-adaf-288ec9d17e23 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:00:56.146919 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-95fbd59a-d541-4dfa-adaf-288ec9d17e23 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:00:56.191797 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fbd59a-d541-4dfa-adaf-288ec9d17e23 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:00:56.193832 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fbd59a-d541-4dfa-adaf-288ec9d17e23 service placement] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:00:56.224225 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 856/1675] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jun 23 16:00:55 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 465 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 16:00:56.243800 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:00:56.273700 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845ed9a0-6aa0-43a7-be69-2cb422ac2c54 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:00:56.289145 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25694a22-6595-43f0-88fc-e3d386e57502 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce18eaaba3a6477684fff1fd98ec8b51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:00:56.290517 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25694a22-6595-43f0-88fc-e3d386e57502 service cinder] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:00:56.290517 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25694a22-6595-43f0-88fc-e3d386e57502 service cinder] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:00:56.290517 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25694a22-6595-43f0-88fc-e3d386e57502 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:00:56.322556 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25694a22-6595-43f0-88fc-e3d386e57502 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:00:56.325201 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25694a22-6595-43f0-88fc-e3d386e57502 service cinder] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:00:56.368831 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 820/1676] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jun 23 16:00:56 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 249 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 16:00:56.484757 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fbd59a-d541-4dfa-adaf-288ec9d17e23 service placement] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:00:56.498310 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fbd59a-d541-4dfa-adaf-288ec9d17e23 service placement] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:00:56.506093 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6187b9fa-866e-4752-87ae-3b278f68ba05 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce18eaaba3a6477684fff1fd98ec8b51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:00:56.507064 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6187b9fa-866e-4752-87ae-3b278f68ba05 service cinder] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:00:56.507497 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6187b9fa-866e-4752-87ae-3b278f68ba05 service cinder] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:00:56.508939 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6187b9fa-866e-4752-87ae-3b278f68ba05 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:00:56.759492 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6187b9fa-866e-4752-87ae-3b278f68ba05 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:00:56.761174 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6187b9fa-866e-4752-87ae-3b278f68ba05 service cinder] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:00:56.805608 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 857/1677] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jun 23 16:00:56 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 16:01:01.430478 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3e1cd8-3633-4554-ad4d-ce112d0a875c None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:01.431434 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3e1cd8-3633-4554-ad4d-ce112d0a875c None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:01.431994 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3e1cd8-3633-4554-ad4d-ce112d0a875c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:01.515581 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-da3e1cd8-3633-4554-ad4d-ce112d0a875c None None] MFA Rules not processed for user `228e5acf4578494ab1a28e4f530233f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:01:01.518784 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-da3e1cd8-3633-4554-ad4d-ce112d0a875c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:01:01.742530 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3317b69a-9f51-486a-bd3e-d82a4f978c75 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:01.743300 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3317b69a-9f51-486a-bd3e-d82a4f978c75 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:01.743300 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3317b69a-9f51-486a-bd3e-d82a4f978c75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:01.753685 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 821/1678] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:01:01 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 324 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 16:01:01.775213 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-3317b69a-9f51-486a-bd3e-d82a4f978c75 None None] MFA Rules not processed for user `228e5acf4578494ab1a28e4f530233f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:01:01.778167 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-3317b69a-9f51-486a-bd3e-d82a4f978c75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:01:01.793260 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 858/1679] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:01:01 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 16:01:01.802831 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3e1cd8-3633-4554-ad4d-ce112d0a875c None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:01:01.805765 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-da3e1cd8-3633-4554-ad4d-ce112d0a875c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:01:01.939624 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3e1cd8-3633-4554-ad4d-ce112d0a875c None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:01:01.958950 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e38aa03-e4bf-4614-bfc6-2c4ee2b7ed1a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '705115d19bab424d894bea8b1b48a7f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:01:01.960431 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e38aa03-e4bf-4614-bfc6-2c4ee2b7ed1a service nova] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:01.960737 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e38aa03-e4bf-4614-bfc6-2c4ee2b7ed1a service nova] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:01.961019 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e38aa03-e4bf-4614-bfc6-2c4ee2b7ed1a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:01.972536 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e38aa03-e4bf-4614-bfc6-2c4ee2b7ed1a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:01:01.975216 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e38aa03-e4bf-4614-bfc6-2c4ee2b7ed1a service nova] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:01:02.000879 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 822/1680] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jun 23 16:01:01 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 16:01:02.053977 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f19a5a6-8330-4408-b2e6-6b633fd8fa04 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:02.054320 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f19a5a6-8330-4408-b2e6-6b633fd8fa04 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:02.054547 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f19a5a6-8330-4408-b2e6-6b633fd8fa04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:02.131315 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-9f19a5a6-8330-4408-b2e6-6b633fd8fa04 None None] MFA Rules not processed for user `5dd4357556944c7089efb34ee58a8e04`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:01:02.137773 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-9f19a5a6-8330-4408-b2e6-6b633fd8fa04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:01:02.297247 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 859/1681] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:01:02 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 16:01:02.530077 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e38aa03-e4bf-4614-bfc6-2c4ee2b7ed1a service nova] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:01:02.532515 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-5e38aa03-e4bf-4614-bfc6-2c4ee2b7ed1a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:01:02.574502 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e38aa03-e4bf-4614-bfc6-2c4ee2b7ed1a service nova] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:01:02.585648 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5b64e3-6596-43f2-8ddb-5021c05d2191 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '705115d19bab424d894bea8b1b48a7f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:01:02.587258 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5b64e3-6596-43f2-8ddb-5021c05d2191 service nova] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:02.589943 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5b64e3-6596-43f2-8ddb-5021c05d2191 service nova] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:02.590481 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5b64e3-6596-43f2-8ddb-5021c05d2191 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:02.636961 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5b64e3-6596-43f2-8ddb-5021c05d2191 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:01:02.645231 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5b64e3-6596-43f2-8ddb-5021c05d2191 service nova] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:01:02.682764 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 823/1682] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jun 23 16:01:02 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 16:01:02.914260 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a66b1b-248e-48f6-affc-cea707e47192 None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:02.914856 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a66b1b-248e-48f6-affc-cea707e47192 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:02.914856 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a66b1b-248e-48f6-affc-cea707e47192 None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:02.917001 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 860/1683] 10.222.0.46 () {54 vars in 912 bytes} [Thu Jun 23 16:01:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 16:01:02.928647 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8921de7b-7e71-4f4b-9435-e12ff6aba3ee None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:02.929153 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8921de7b-7e71-4f4b-9435-e12ff6aba3ee None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:02.932371 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8921de7b-7e71-4f4b-9435-e12ff6aba3ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:02.992698 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-8921de7b-7e71-4f4b-9435-e12ff6aba3ee None None] MFA Rules not processed for user `51784064951a4619ae09b9167fb0fd0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:01:02.996098 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-8921de7b-7e71-4f4b-9435-e12ff6aba3ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:01:03.115142 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:01:03.145396 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 824/1684] 10.222.0.46 () {58 vars in 1009 bytes} [Thu Jun 23 16:01:02 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 16:01:03.165070 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a66b1b-248e-48f6-affc-cea707e47192 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:01:03.167133 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-17a66b1b-248e-48f6-affc-cea707e47192 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:01:03.186991 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a66b1b-248e-48f6-affc-cea707e47192 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:01:03.200949 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4408975-3de3-4cf4-9519-4d39e18fc18d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51784064951a4619ae09b9167fb0fd0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:01:03.201704 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4408975-3de3-4cf4-9519-4d39e18fc18d service neutron] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:03.201993 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4408975-3de3-4cf4-9519-4d39e18fc18d service neutron] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:03.202269 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4408975-3de3-4cf4-9519-4d39e18fc18d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:03.226436 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4408975-3de3-4cf4-9519-4d39e18fc18d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:01:03.230411 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4408975-3de3-4cf4-9519-4d39e18fc18d service neutron] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:01:03.254175 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 861/1685] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jun 23 16:01:03 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 16:01:03.351554 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63d13f2-33b1-4493-a85d-a0cdbcf07467 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:03.352461 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63d13f2-33b1-4493-a85d-a0cdbcf07467 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:03.352921 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63d13f2-33b1-4493-a85d-a0cdbcf07467 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:03.388710 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-d63d13f2-33b1-4493-a85d-a0cdbcf07467 None None] MFA Rules not processed for user `dac5e323b42f41859c85709a2b49623b`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:01:03.390529 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-d63d13f2-33b1-4493-a85d-a0cdbcf07467 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:01:03.445097 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f211eb-442e-438b-9a51-f272cb748208 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:03.445615 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f211eb-442e-438b-9a51-f272cb748208 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:03.446116 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f211eb-442e-438b-9a51-f272cb748208 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:03.531784 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-59f211eb-442e-438b-9a51-f272cb748208 None None] MFA Rules not processed for user `be48addfaa154555b8d0cb7aac8b32ac`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:01:03.547116 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-59f211eb-442e-438b-9a51-f272cb748208 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:01:03.582537 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 825/1686] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:01:03 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 16:01:03.762776 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63d13f2-33b1-4493-a85d-a0cdbcf07467 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:01:03.809755 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63d13f2-33b1-4493-a85d-a0cdbcf07467 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:01:03.847936 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:01:03.852628 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ffa327-1511-4699-b53e-672b7fbffe85 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '705115d19bab424d894bea8b1b48a7f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:01:03.854302 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ffa327-1511-4699-b53e-672b7fbffe85 service nova] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:03.859360 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ffa327-1511-4699-b53e-672b7fbffe85 service nova] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:03.860024 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ffa327-1511-4699-b53e-672b7fbffe85 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:03.923179 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 862/1687] 10.222.0.46 () {58 vars in 1009 bytes} [Thu Jun 23 16:01:03 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 478 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 16:01:03.929579 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ffa327-1511-4699-b53e-672b7fbffe85 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:01:03.941087 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ffa327-1511-4699-b53e-672b7fbffe85 service nova] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:01:03.999253 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 826/1688] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jun 23 16:01:03 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 16:01:04.236721 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f211eb-442e-438b-9a51-f272cb748208 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:01:04.238715 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-59f211eb-442e-438b-9a51-f272cb748208 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:01:04.552895 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f211eb-442e-438b-9a51-f272cb748208 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:01:04.568669 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aead39b-b184-463b-8da5-d63f6a745483 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '242bf02d14724fe4837fcce8730377f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:01:04.569676 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aead39b-b184-463b-8da5-d63f6a745483 service swift] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:04.570772 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aead39b-b184-463b-8da5-d63f6a745483 service swift] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:04.575367 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aead39b-b184-463b-8da5-d63f6a745483 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:04.596321 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aead39b-b184-463b-8da5-d63f6a745483 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:01:04.598630 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aead39b-b184-463b-8da5-d63f6a745483 service swift] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:01:04.649257 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 863/1689] 10.222.0.46 () {60 vars in 1384 bytes} [Thu Jun 23 16:01:04 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 23 16:01:05.769796 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ffa327-1511-4699-b53e-672b7fbffe85 service nova] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:01:05.771690 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-d1ffa327-1511-4699-b53e-672b7fbffe85 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:01:06.027108 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ffa327-1511-4699-b53e-672b7fbffe85 service nova] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:01:06.041733 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18310cf-7449-4259-94dc-db136537b1d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:01:06.042722 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18310cf-7449-4259-94dc-db136537b1d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:06.043071 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18310cf-7449-4259-94dc-db136537b1d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:06.043385 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18310cf-7449-4259-94dc-db136537b1d4 admin admin] PATH_INFO: `/v3/users/d887ce5851b5410094f1bd83120935c1` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:06.048174 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18310cf-7449-4259-94dc-db136537b1d4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d887ce5851b5410094f1bd83120935c1)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:01:06.055532 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18310cf-7449-4259-94dc-db136537b1d4 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:01:06.171153 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-f18310cf-7449-4259-94dc-db136537b1d4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd887ce5851b5410094f1bd83120935c1'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:01:06.174547 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-f18310cf-7449-4259-94dc-db136537b1d4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd887ce5851b5410094f1bd83120935c1'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:01:06.195536 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-f18310cf-7449-4259-94dc-db136537b1d4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd887ce5851b5410094f1bd83120935c1'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:01:06.217979 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-f18310cf-7449-4259-94dc-db136537b1d4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd887ce5851b5410094f1bd83120935c1'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:01:06.219196 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-f18310cf-7449-4259-94dc-db136537b1d4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd887ce5851b5410094f1bd83120935c1'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:01:06.240355 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:01:06.280706 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36340b52-4e57-4467-b945-895ecf324dda None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:06.281684 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36340b52-4e57-4467-b945-895ecf324dda None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:06.282115 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36340b52-4e57-4467-b945-895ecf324dda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:06.283671 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 827/1690] 10.222.0.46 () {58 vars in 1223 bytes} [Thu Jun 23 16:01:05 2022] DELETE /identity/v3/users/d887ce5851b5410094f1bd83120935c1 => generated 0 bytes in 507 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 16:01:06.339576 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-36340b52-4e57-4467-b945-895ecf324dda None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:01:06.345299 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-36340b52-4e57-4467-b945-895ecf324dda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:01:06.612425 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 864/1691] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:01:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 333 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 16:01:06.702767 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13197cc0-11b5-47fb-a666-e438d279d6e7 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:06.703077 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13197cc0-11b5-47fb-a666-e438d279d6e7 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:06.704486 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13197cc0-11b5-47fb-a666-e438d279d6e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:06.767959 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-13197cc0-11b5-47fb-a666-e438d279d6e7 None None] MFA Rules not processed for user `228e5acf4578494ab1a28e4f530233f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:01:06.769231 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-13197cc0-11b5-47fb-a666-e438d279d6e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:01:06.855654 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36340b52-4e57-4467-b945-895ecf324dda None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:01:06.859413 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-36340b52-4e57-4467-b945-895ecf324dda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:01:07.098205 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 828/1692] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:01:06 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 397 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 16:01:07.133128 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36340b52-4e57-4467-b945-895ecf324dda None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:01:07.165622 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e3e34a-627a-469a-ba05-dea9b7cd0feb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51784064951a4619ae09b9167fb0fd0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:01:07.166942 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e3e34a-627a-469a-ba05-dea9b7cd0feb service neutron] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:07.167246 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e3e34a-627a-469a-ba05-dea9b7cd0feb service neutron] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:07.167523 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e3e34a-627a-469a-ba05-dea9b7cd0feb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:07.221349 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e3e34a-627a-469a-ba05-dea9b7cd0feb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:01:07.223443 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e3e34a-627a-469a-ba05-dea9b7cd0feb service neutron] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:01:07.255866 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 865/1693] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jun 23 16:01:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 16:01:09.403785 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13197cc0-11b5-47fb-a666-e438d279d6e7 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:01:09.405415 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-13197cc0-11b5-47fb-a666-e438d279d6e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:01:09.428538 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13197cc0-11b5-47fb-a666-e438d279d6e7 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:01:09.442377 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdde80a-3f91-4088-a99f-7b5a3488949c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:01:09.443971 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdde80a-3f91-4088-a99f-7b5a3488949c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:09.444328 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdde80a-3f91-4088-a99f-7b5a3488949c admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:09.444680 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdde80a-3f91-4088-a99f-7b5a3488949c admin admin] PATH_INFO: `/v3/projects/80d7b052110b4d61afa8de89a996b580` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:09.446519 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfdde80a-3f91-4088-a99f-7b5a3488949c admin admin] RBAC: Authorizing `identity:delete_project(project_id=80d7b052110b4d61afa8de89a996b580)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:01:09.449241 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfdde80a-3f91-4088-a99f-7b5a3488949c admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:01:09.600529 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-bfdde80a-3f91-4088-a99f-7b5a3488949c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '80d7b052110b4d61afa8de89a996b580', 'request_id': 'req-bfdde80a-3f91-4088-a99f-7b5a3488949c'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:01:09.619623 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-bfdde80a-3f91-4088-a99f-7b5a3488949c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '80d7b052110b4d61afa8de89a996b580', 'request_id': 'req-bfdde80a-3f91-4088-a99f-7b5a3488949c'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:01:09.643533 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-bfdde80a-3f91-4088-a99f-7b5a3488949c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '80d7b052110b4d61afa8de89a996b580', 'request_id': 'req-bfdde80a-3f91-4088-a99f-7b5a3488949c'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:01:09.677056 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-bfdde80a-3f91-4088-a99f-7b5a3488949c admin admin] The token cache is being invalidate because project 80d7b052110b4d61afa8de89a996b580 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80154) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 16:01:09.677358 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-bfdde80a-3f91-4088-a99f-7b5a3488949c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:01:09.678123 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-bfdde80a-3f91-4088-a99f-7b5a3488949c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:01:09.683934 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 829/1694] 10.222.0.46 () {58 vars in 1232 bytes} [Thu Jun 23 16:01:09 2022] DELETE /identity/v3/projects/80d7b052110b4d61afa8de89a996b580 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 16:01:09.685556 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119c139c-6cfc-4a57-a653-aad875bc6c78 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:09.686081 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119c139c-6cfc-4a57-a653-aad875bc6c78 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:09.686162 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119c139c-6cfc-4a57-a653-aad875bc6c78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:09.746915 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-119c139c-6cfc-4a57-a653-aad875bc6c78 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:01:09.748699 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-119c139c-6cfc-4a57-a653-aad875bc6c78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:01:09.825127 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:01:09.949254 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 866/1695] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:01:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 16:01:09.953370 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea29f531-dd9c-41ae-b76b-cb0518123f90 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:09.953662 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea29f531-dd9c-41ae-b76b-cb0518123f90 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:09.953948 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea29f531-dd9c-41ae-b76b-cb0518123f90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:09.994894 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-ea29f531-dd9c-41ae-b76b-cb0518123f90 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:01:09.996254 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-ea29f531-dd9c-41ae-b76b-cb0518123f90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:01:10.046802 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 830/1696] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:01:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 16:01:10.050724 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119c139c-6cfc-4a57-a653-aad875bc6c78 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:01:10.078720 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119c139c-6cfc-4a57-a653-aad875bc6c78 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:01:10.097741 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7461278e-f3aa-424b-89e6-44381700f796 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:01:10.099153 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7461278e-f3aa-424b-89e6-44381700f796 admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:10.099591 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7461278e-f3aa-424b-89e6-44381700f796 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:10.100036 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7461278e-f3aa-424b-89e6-44381700f796 admin admin] PATH_INFO: `/v3/domains` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:10.101360 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7461278e-f3aa-424b-89e6-44381700f796 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 16:01:10.101835 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7461278e-f3aa-424b-89e6-44381700f796 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:01:10.109459 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7461278e-f3aa-424b-89e6-44381700f796 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:01:10.141799 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 867/1697] 10.222.0.46 () {58 vars in 1152 bytes} [Thu Jun 23 16:01:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 16:01:10.146646 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea29f531-dd9c-41ae-b76b-cb0518123f90 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:01:10.162409 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea29f531-dd9c-41ae-b76b-cb0518123f90 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:01:10.173454 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edaff5ad-14ec-4c10-a86c-5625059e5929 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:01:10.174255 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edaff5ad-14ec-4c10-a86c-5625059e5929 admin admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:10.174536 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edaff5ad-14ec-4c10-a86c-5625059e5929 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:10.175312 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edaff5ad-14ec-4c10-a86c-5625059e5929 admin admin] PATH_INFO: `/v3/projects` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:10.176266 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edaff5ad-14ec-4c10-a86c-5625059e5929 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:01:10.178269 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edaff5ad-14ec-4c10-a86c-5625059e5929 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:01:10.277594 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 831/1698] 10.222.0.46 () {60 vars in 1152 bytes} [Thu Jun 23 16:01:10 2022] POST /identity/v3/projects => generated 371 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 16:01:10.278416 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7461278e-f3aa-424b-89e6-44381700f796 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:01:10.304275 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7461278e-f3aa-424b-89e6-44381700f796 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:01:10.494322 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f18fe22-d67c-49d8-9758-912d06cd6ecf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:01:10.495139 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f18fe22-d67c-49d8-9758-912d06cd6ecf admin admin] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:10.495461 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f18fe22-d67c-49d8-9758-912d06cd6ecf admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:10.495743 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f18fe22-d67c-49d8-9758-912d06cd6ecf admin admin] PATH_INFO: `/v3/users` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:10.496483 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f18fe22-d67c-49d8-9758-912d06cd6ecf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:01:10.498673 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f18fe22-d67c-49d8-9758-912d06cd6ecf admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:01:10.585568 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 868/1699] 10.222.0.46 () {60 vars in 1143 bytes} [Thu Jun 23 16:01:10 2022] POST /identity/v3/users => generated 286 bytes in 305 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 16:01:10.587065 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edaff5ad-14ec-4c10-a86c-5625059e5929 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:01:10.608602 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edaff5ad-14ec-4c10-a86c-5625059e5929 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:01:10.636221 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5c457f-562d-4041-ac23-08df08f15ece admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:01:10.636956 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5c457f-562d-4041-ac23-08df08f15ece admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:10.637247 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5c457f-562d-4041-ac23-08df08f15ece admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:10.637534 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5c457f-562d-4041-ac23-08df08f15ece admin admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:10.638407 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5c457f-562d-4041-ac23-08df08f15ece admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:01:10.640214 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5c457f-562d-4041-ac23-08df08f15ece admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:01:10.694616 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 832/1700] 10.222.0.46 () {58 vars in 1121 bytes} [Thu Jun 23 16:01:10 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 16:01:10.696290 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f18fe22-d67c-49d8-9758-912d06cd6ecf admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:01:10.760490 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f18fe22-d67c-49d8-9758-912d06cd6ecf admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:01:10.787495 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065b365d-f840-4342-b304-b40b13e3ab09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:01:10.788710 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065b365d-f840-4342-b304-b40b13e3ab09 admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:10.791216 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065b365d-f840-4342-b304-b40b13e3ab09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:10.791721 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065b365d-f840-4342-b304-b40b13e3ab09 admin admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:10.792670 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065b365d-f840-4342-b304-b40b13e3ab09 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:01:10.800732 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065b365d-f840-4342-b304-b40b13e3ab09 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:01:10.843634 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 869/1701] 10.222.0.46 () {58 vars in 1121 bytes} [Thu Jun 23 16:01:10 2022] GET /identity/v3/roles => generated 1420 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 16:01:10.845742 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5c457f-562d-4041-ac23-08df08f15ece admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:01:10.866574 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5c457f-562d-4041-ac23-08df08f15ece admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:01:10.888134 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5370c3-9f84-493a-a619-1a37c657b018 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:01:10.889081 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5370c3-9f84-493a-a619-1a37c657b018 admin admin] REQUEST_METHOD: `PUT` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:10.889368 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5370c3-9f84-493a-a619-1a37c657b018 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:10.889676 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5370c3-9f84-493a-a619-1a37c657b018 admin admin] PATH_INFO: `/v3/projects/2975413ebd8643f8aba67386288cb006/users/6f438585929c424b92f183dd8cc7b5db/roles/e2a5bbf3200a4d4091d72bc7ff78c34d` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:10.955577 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5370c3-9f84-493a-a619-1a37c657b018 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2975413ebd8643f8aba67386288cb006, user_id=6f438585929c424b92f183dd8cc7b5db, role_id=e2a5bbf3200a4d4091d72bc7ff78c34d)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:01:10.958480 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5370c3-9f84-493a-a619-1a37c657b018 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:01:11.009308 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 833/1702] 10.222.0.46 () {60 vars in 1482 bytes} [Thu Jun 23 16:01:10 2022] PUT /identity/v3/projects/2975413ebd8643f8aba67386288cb006/users/6f438585929c424b92f183dd8cc7b5db/roles/e2a5bbf3200a4d4091d72bc7ff78c34d => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 16:01:11.012241 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbb2afc-46b2-468d-acc3-42aaf494ad1f None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:11.012719 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbb2afc-46b2-468d-acc3-42aaf494ad1f None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:11.013147 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbb2afc-46b2-468d-acc3-42aaf494ad1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:11.013578 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d588e2ed-d1d3-4e08-a084-5c9dacef026b None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:11.014169 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d588e2ed-d1d3-4e08-a084-5c9dacef026b None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:11.014726 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d588e2ed-d1d3-4e08-a084-5c9dacef026b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:11.068032 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-0cbb2afc-46b2-468d-acc3-42aaf494ad1f None None] MFA Rules not processed for user `5dd4357556944c7089efb34ee58a8e04`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:01:11.069310 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-0cbb2afc-46b2-468d-acc3-42aaf494ad1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:01:11.106107 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-d588e2ed-d1d3-4e08-a084-5c9dacef026b None None] MFA Rules not processed for user `6f438585929c424b92f183dd8cc7b5db`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:01:11.108278 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-d588e2ed-d1d3-4e08-a084-5c9dacef026b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:01:11.120145 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=89,keystone:DELETE=8,keystone:INSERT=3 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:01:11.377891 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 834/1703] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:01:11 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 363 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 16:01:11.377891 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d021a50-a74e-46c3-857f-ea8b48d4fa34 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:11.377891 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d021a50-a74e-46c3-857f-ea8b48d4fa34 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:11.377891 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d021a50-a74e-46c3-857f-ea8b48d4fa34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:11.388124 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 870/1704] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:01:11 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 376 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 16:01:11.393761 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbb2afc-46b2-468d-acc3-42aaf494ad1f None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:01:11.441604 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-6d021a50-a74e-46c3-857f-ea8b48d4fa34 None None] MFA Rules not processed for user `51784064951a4619ae09b9167fb0fd0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:01:11.455751 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-6d021a50-a74e-46c3-857f-ea8b48d4fa34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:01:11.609257 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:01:11.691880 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbb2afc-46b2-468d-acc3-42aaf494ad1f None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:01:11.715572 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea611c15-2d48-4c06-b514-e732d9147b1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:01:11.716421 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea611c15-2d48-4c06-b514-e732d9147b1c admin admin] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:11.716921 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea611c15-2d48-4c06-b514-e732d9147b1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:11.717529 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea611c15-2d48-4c06-b514-e732d9147b1c admin admin] PATH_INFO: `/v3/projects` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:11.718709 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea611c15-2d48-4c06-b514-e732d9147b1c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:01:11.721601 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea611c15-2d48-4c06-b514-e732d9147b1c admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:01:11.811690 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 835/1705] 10.222.0.44 () {58 vars in 1015 bytes} [Thu Jun 23 16:01:11 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 437 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 16:01:11.950812 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 871/1706] 10.222.0.46 () {60 vars in 1152 bytes} [Thu Jun 23 16:01:11 2022] POST /identity/v3/projects => generated 373 bytes in 562 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 16:01:11.956081 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d021a50-a74e-46c3-857f-ea8b48d4fa34 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:01:11.988307 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d021a50-a74e-46c3-857f-ea8b48d4fa34 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:01:12.226231 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f5097a-88b7-4e64-9386-45ad512861ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:01:12.228626 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f5097a-88b7-4e64-9386-45ad512861ab admin admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:12.228991 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f5097a-88b7-4e64-9386-45ad512861ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:12.243236 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f5097a-88b7-4e64-9386-45ad512861ab admin admin] PATH_INFO: `/v3/users` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:12.266278 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f5097a-88b7-4e64-9386-45ad512861ab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:01:12.272414 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f5097a-88b7-4e64-9386-45ad512861ab admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:01:12.377018 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 836/1707] 10.222.0.46 () {60 vars in 1143 bytes} [Thu Jun 23 16:01:11 2022] POST /identity/v3/users => generated 293 bytes in 419 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 16:01:12.379282 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea611c15-2d48-4c06-b514-e732d9147b1c admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:01:12.405179 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea611c15-2d48-4c06-b514-e732d9147b1c admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:01:12.431067 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8984e6c-ff8b-4cb2-8599-6edb21cf7731 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:01:12.432360 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8984e6c-ff8b-4cb2-8599-6edb21cf7731 admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:12.432677 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8984e6c-ff8b-4cb2-8599-6edb21cf7731 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:12.432937 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8984e6c-ff8b-4cb2-8599-6edb21cf7731 admin admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:12.434080 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8984e6c-ff8b-4cb2-8599-6edb21cf7731 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:01:12.445948 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8984e6c-ff8b-4cb2-8599-6edb21cf7731 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:01:12.501815 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f5097a-88b7-4e64-9386-45ad512861ab admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:01:12.509338 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 872/1708] 10.222.0.46 () {58 vars in 1121 bytes} [Thu Jun 23 16:01:12 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 16:01:12.534938 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f5097a-88b7-4e64-9386-45ad512861ab admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:01:12.580845 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc13b2e-b4c1-46c5-bf70-568c006af08d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:01:12.581967 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc13b2e-b4c1-46c5-bf70-568c006af08d admin admin] REQUEST_METHOD: `PUT` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:12.582436 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc13b2e-b4c1-46c5-bf70-568c006af08d admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:12.584622 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc13b2e-b4c1-46c5-bf70-568c006af08d admin admin] PATH_INFO: `/v3/projects/a3cb13a757a746a2915ca1e8d9ad12b0/users/8f76af88aa0b40fab5833df2d32b6f54/roles/812d3c567c52469cb2b35709fbdc857c` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:12.631562 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc13b2e-b4c1-46c5-bf70-568c006af08d admin admin] RBAC: Authorizing `identity:create_grant(project_id=a3cb13a757a746a2915ca1e8d9ad12b0, user_id=8f76af88aa0b40fab5833df2d32b6f54, role_id=812d3c567c52469cb2b35709fbdc857c)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:01:12.637738 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc13b2e-b4c1-46c5-bf70-568c006af08d admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:01:12.670111 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8984e6c-ff8b-4cb2-8599-6edb21cf7731 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:01:12.670323 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 837/1709] 10.222.0.46 () {60 vars in 1482 bytes} [Thu Jun 23 16:01:12 2022] PUT /identity/v3/projects/a3cb13a757a746a2915ca1e8d9ad12b0/users/8f76af88aa0b40fab5833df2d32b6f54/roles/812d3c567c52469cb2b35709fbdc857c => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 16:01:12.674734 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc13b2e-b4c1-46c5-bf70-568c006af08d admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:01:12.678086 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-c8984e6c-ff8b-4cb2-8599-6edb21cf7731 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:01:12.784258 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:01:13.167670 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8984e6c-ff8b-4cb2-8599-6edb21cf7731 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:01:13.195146 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc13b2e-b4c1-46c5-bf70-568c006af08d admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:01:13.221500 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cf7c8f-672d-4df7-b618-5aa60bdf5829 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:01:13.236856 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cf7c8f-672d-4df7-b618-5aa60bdf5829 admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:13.239051 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cf7c8f-672d-4df7-b618-5aa60bdf5829 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:13.240425 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cf7c8f-672d-4df7-b618-5aa60bdf5829 admin admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:13.243670 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cf7c8f-672d-4df7-b618-5aa60bdf5829 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:01:13.251611 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cf7c8f-672d-4df7-b618-5aa60bdf5829 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:01:13.287775 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 838/1710] 10.222.0.46 () {58 vars in 1121 bytes} [Thu Jun 23 16:01:12 2022] GET /identity/v3/roles => generated 1420 bytes in 614 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 16:01:13.291652 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa886b58-1669-4d29-b400-0b3b94bfa0b0 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:13.291944 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa886b58-1669-4d29-b400-0b3b94bfa0b0 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:13.292231 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa886b58-1669-4d29-b400-0b3b94bfa0b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:13.352144 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-fa886b58-1669-4d29-b400-0b3b94bfa0b0 None None] MFA Rules not processed for user `dac5e323b42f41859c85709a2b49623b`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:01:13.354768 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-fa886b58-1669-4d29-b400-0b3b94bfa0b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:01:13.379375 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907245a6-8e0d-44ea-a4ba-d0c530e03328 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51784064951a4619ae09b9167fb0fd0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:01:13.380898 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907245a6-8e0d-44ea-a4ba-d0c530e03328 service neutron] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:13.381636 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907245a6-8e0d-44ea-a4ba-d0c530e03328 service neutron] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:13.382671 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907245a6-8e0d-44ea-a4ba-d0c530e03328 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:13.421746 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907245a6-8e0d-44ea-a4ba-d0c530e03328 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:01:13.426815 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907245a6-8e0d-44ea-a4ba-d0c530e03328 service neutron] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:01:13.453873 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 873/1711] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jun 23 16:01:12 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 785 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 16:01:13.456356 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907245a6-8e0d-44ea-a4ba-d0c530e03328 service neutron] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:01:13.495309 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907245a6-8e0d-44ea-a4ba-d0c530e03328 service neutron] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:01:13.511025 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7a8532-87f9-4970-96e4-cf6ebb833d90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:01:13.513698 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7a8532-87f9-4970-96e4-cf6ebb833d90 admin admin] REQUEST_METHOD: `PUT` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:13.514022 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7a8532-87f9-4970-96e4-cf6ebb833d90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:13.516574 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7a8532-87f9-4970-96e4-cf6ebb833d90 admin admin] PATH_INFO: `/v3/projects/a3cb13a757a746a2915ca1e8d9ad12b0/users/8f76af88aa0b40fab5833df2d32b6f54/roles/812d3c567c52469cb2b35709fbdc857c` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:13.522635 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7a8532-87f9-4970-96e4-cf6ebb833d90 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a3cb13a757a746a2915ca1e8d9ad12b0, user_id=8f76af88aa0b40fab5833df2d32b6f54, role_id=812d3c567c52469cb2b35709fbdc857c)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:01:13.530437 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7a8532-87f9-4970-96e4-cf6ebb833d90 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:01:13.557219 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 839/1712] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:01:13 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 16:01:13.595813 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 874/1713] 10.222.0.46 () {60 vars in 1482 bytes} [Thu Jun 23 16:01:13 2022] PUT /identity/v3/projects/a3cb13a757a746a2915ca1e8d9ad12b0/users/8f76af88aa0b40fab5833df2d32b6f54/roles/812d3c567c52469cb2b35709fbdc857c => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 16:01:13.611135 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048bb7dc-6364-43c9-9d2c-68696a90bfa9 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:13.613649 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048bb7dc-6364-43c9-9d2c-68696a90bfa9 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:13.614656 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048bb7dc-6364-43c9-9d2c-68696a90bfa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:13.744621 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-048bb7dc-6364-43c9-9d2c-68696a90bfa9 None None] MFA Rules not processed for user `8f76af88aa0b40fab5833df2d32b6f54`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:01:13.766177 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-048bb7dc-6364-43c9-9d2c-68696a90bfa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:01:14.010954 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 840/1714] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:01:13 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 405 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 16:01:14.031874 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7a8532-87f9-4970-96e4-cf6ebb833d90 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:01:14.033950 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-2f7a8532-87f9-4970-96e4-cf6ebb833d90 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:01:14.205156 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7a8532-87f9-4970-96e4-cf6ebb833d90 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:01:14.215999 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd916eb5-f614-48de-881b-735b293f764b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce18eaaba3a6477684fff1fd98ec8b51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:01:14.215999 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd916eb5-f614-48de-881b-735b293f764b service cinder] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:14.215999 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd916eb5-f614-48de-881b-735b293f764b service cinder] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:14.215999 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd916eb5-f614-48de-881b-735b293f764b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:14.244732 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd916eb5-f614-48de-881b-735b293f764b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:01:14.251416 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd916eb5-f614-48de-881b-735b293f764b service cinder] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:01:14.273586 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 875/1715] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jun 23 16:01:14 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 242 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 16:01:14.864239 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048bb7dc-6364-43c9-9d2c-68696a90bfa9 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:01:14.866560 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-048bb7dc-6364-43c9-9d2c-68696a90bfa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:01:15.107517 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:01:15.153272 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048bb7dc-6364-43c9-9d2c-68696a90bfa9 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:01:15.170749 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe736b9-a070-4b2b-8247-a174b543f3f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:01:15.175422 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe736b9-a070-4b2b-8247-a174b543f3f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:15.176520 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe736b9-a070-4b2b-8247-a174b543f3f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:15.179602 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe736b9-a070-4b2b-8247-a174b543f3f6 admin admin] PATH_INFO: `/v3/users/3536ffe4f5194e78b4edee59471752b0` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:15.188555 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe736b9-a070-4b2b-8247-a174b543f3f6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3536ffe4f5194e78b4edee59471752b0)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:01:15.195521 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe736b9-a070-4b2b-8247-a174b543f3f6 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:01:15.364151 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-cfe736b9-a070-4b2b-8247-a174b543f3f6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3536ffe4f5194e78b4edee59471752b0'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:01:15.365783 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-cfe736b9-a070-4b2b-8247-a174b543f3f6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3536ffe4f5194e78b4edee59471752b0'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:01:15.386614 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-cfe736b9-a070-4b2b-8247-a174b543f3f6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3536ffe4f5194e78b4edee59471752b0'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:01:15.415080 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-cfe736b9-a070-4b2b-8247-a174b543f3f6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3536ffe4f5194e78b4edee59471752b0'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:01:15.416272 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-cfe736b9-a070-4b2b-8247-a174b543f3f6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3536ffe4f5194e78b4edee59471752b0'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:01:15.459176 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 841/1716] 10.222.0.46 () {58 vars in 1223 bytes} [Thu Jun 23 16:01:14 2022] DELETE /identity/v3/users/3536ffe4f5194e78b4edee59471752b0 => generated 0 bytes in 593 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 16:01:15.460758 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394b9f62-8dd8-4edd-93bd-a00ffffa1180 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:15.461279 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394b9f62-8dd8-4edd-93bd-a00ffffa1180 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:15.461370 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394b9f62-8dd8-4edd-93bd-a00ffffa1180 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:15.471671 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:01:15.507968 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-394b9f62-8dd8-4edd-93bd-a00ffffa1180 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:01:15.513485 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-394b9f62-8dd8-4edd-93bd-a00ffffa1180 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:01:15.725419 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 876/1717] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:01:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 16:01:15.733390 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe736b9-a070-4b2b-8247-a174b543f3f6 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:01:15.734640 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-cfe736b9-a070-4b2b-8247-a174b543f3f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:01:15.769102 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe736b9-a070-4b2b-8247-a174b543f3f6 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:01:15.795586 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90756c08-df7d-4af9-b7fc-148bb92c2eda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:01:15.796733 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90756c08-df7d-4af9-b7fc-148bb92c2eda admin admin] REQUEST_METHOD: `DELETE` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:15.797105 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90756c08-df7d-4af9-b7fc-148bb92c2eda admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:15.797475 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90756c08-df7d-4af9-b7fc-148bb92c2eda admin admin] PATH_INFO: `/v3/users/6f438585929c424b92f183dd8cc7b5db` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:15.801390 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90756c08-df7d-4af9-b7fc-148bb92c2eda admin admin] RBAC: Authorizing `identity:delete_user(user_id=6f438585929c424b92f183dd8cc7b5db)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:01:15.805297 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90756c08-df7d-4af9-b7fc-148bb92c2eda admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:01:15.864538 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394b9f62-8dd8-4edd-93bd-a00ffffa1180 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:01:15.866173 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-394b9f62-8dd8-4edd-93bd-a00ffffa1180 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:01:15.901807 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394b9f62-8dd8-4edd-93bd-a00ffffa1180 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:01:15.916799 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bdacb2-3e4d-4b2d-a49f-4730ad8c3701 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:01:15.920700 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bdacb2-3e4d-4b2d-a49f-4730ad8c3701 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:15.921592 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bdacb2-3e4d-4b2d-a49f-4730ad8c3701 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:15.922592 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bdacb2-3e4d-4b2d-a49f-4730ad8c3701 admin admin] PATH_INFO: `/v3/users/3263967dbd554d7bb70c8c0b074ff247` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:15.926962 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bdacb2-3e4d-4b2d-a49f-4730ad8c3701 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3263967dbd554d7bb70c8c0b074ff247)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:01:15.932954 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bdacb2-3e4d-4b2d-a49f-4730ad8c3701 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:01:15.963933 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-90756c08-df7d-4af9-b7fc-148bb92c2eda admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6f438585929c424b92f183dd8cc7b5db'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:01:15.966083 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-90756c08-df7d-4af9-b7fc-148bb92c2eda admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6f438585929c424b92f183dd8cc7b5db'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:01:15.986501 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-90756c08-df7d-4af9-b7fc-148bb92c2eda admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6f438585929c424b92f183dd8cc7b5db'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:01:16.019001 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-90756c08-df7d-4af9-b7fc-148bb92c2eda admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6f438585929c424b92f183dd8cc7b5db'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:01:16.021416 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-90756c08-df7d-4af9-b7fc-148bb92c2eda admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6f438585929c424b92f183dd8cc7b5db'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:01:16.056025 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 842/1718] 10.222.0.46 () {58 vars in 1223 bytes} [Thu Jun 23 16:01:15 2022] DELETE /identity/v3/users/6f438585929c424b92f183dd8cc7b5db => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 16:01:16.058183 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90756c08-df7d-4af9-b7fc-148bb92c2eda admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:01:16.061074 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-90756c08-df7d-4af9-b7fc-148bb92c2eda admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:01:16.070647 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-69bdacb2-3e4d-4b2d-a49f-4730ad8c3701 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3263967dbd554d7bb70c8c0b074ff247'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:01:16.093735 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-69bdacb2-3e4d-4b2d-a49f-4730ad8c3701 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3263967dbd554d7bb70c8c0b074ff247'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:01:16.094541 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-69bdacb2-3e4d-4b2d-a49f-4730ad8c3701 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3263967dbd554d7bb70c8c0b074ff247'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:01:16.095206 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-69bdacb2-3e4d-4b2d-a49f-4730ad8c3701 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3263967dbd554d7bb70c8c0b074ff247'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:01:16.128119 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-69bdacb2-3e4d-4b2d-a49f-4730ad8c3701 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3263967dbd554d7bb70c8c0b074ff247'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:01:16.157582 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 877/1719] 10.222.0.46 () {58 vars in 1223 bytes} [Thu Jun 23 16:01:15 2022] DELETE /identity/v3/users/3263967dbd554d7bb70c8c0b074ff247 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 16:01:16.159209 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1106537b-a060-450c-bbd7-e722f827583d None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:16.159505 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1106537b-a060-450c-bbd7-e722f827583d None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:16.161615 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1106537b-a060-450c-bbd7-e722f827583d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:16.180235 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:01:16.212196 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-1106537b-a060-450c-bbd7-e722f827583d None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:01:16.213652 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-1106537b-a060-450c-bbd7-e722f827583d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:01:16.221544 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90756c08-df7d-4af9-b7fc-148bb92c2eda admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:01:16.348571 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c0f9af-dc03-4d13-8889-3f9653fe2d9c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51784064951a4619ae09b9167fb0fd0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:01:16.349398 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c0f9af-dc03-4d13-8889-3f9653fe2d9c service neutron] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:16.349830 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c0f9af-dc03-4d13-8889-3f9653fe2d9c service neutron] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:16.350217 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c0f9af-dc03-4d13-8889-3f9653fe2d9c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:16.360408 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-d2c0f9af-dc03-4d13-8889-3f9653fe2d9c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:01:16.533736 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 878/1720] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:01:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 376 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 16:01:16.535264 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43eed151-521c-4be9-948e-33b587535928 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:16.537043 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43eed151-521c-4be9-948e-33b587535928 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:16.537355 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43eed151-521c-4be9-948e-33b587535928 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:16.608952 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-43eed151-521c-4be9-948e-33b587535928 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:01:16.611949 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-43eed151-521c-4be9-948e-33b587535928 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:01:16.643135 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 879/1721] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:01:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 16:01:16.646980 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43eed151-521c-4be9-948e-33b587535928 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:01:16.650867 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c0f9af-dc03-4d13-8889-3f9653fe2d9c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:01:16.655795 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c0f9af-dc03-4d13-8889-3f9653fe2d9c service neutron] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:01:16.659116 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-43eed151-521c-4be9-948e-33b587535928 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:01:16.704445 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 843/1722] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jun 23 16:01:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 647 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 16:01:16.720699 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43eed151-521c-4be9-948e-33b587535928 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:01:16.745713 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbbaa0f-6365-4303-aa2e-9eb9fd2ea35d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51784064951a4619ae09b9167fb0fd0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:01:16.746193 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbbaa0f-6365-4303-aa2e-9eb9fd2ea35d service neutron] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:16.746488 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbbaa0f-6365-4303-aa2e-9eb9fd2ea35d service neutron] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:16.750753 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbbaa0f-6365-4303-aa2e-9eb9fd2ea35d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:16.758899 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c0f9af-dc03-4d13-8889-3f9653fe2d9c service neutron] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:01:16.761217 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-d2c0f9af-dc03-4d13-8889-3f9653fe2d9c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:01:16.796946 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbbaa0f-6365-4303-aa2e-9eb9fd2ea35d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:01:16.797260 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c0f9af-dc03-4d13-8889-3f9653fe2d9c service neutron] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:01:16.803658 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbbaa0f-6365-4303-aa2e-9eb9fd2ea35d service neutron] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:01:16.817402 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85e315d-9616-49d3-af9a-9af00ad8b615 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51784064951a4619ae09b9167fb0fd0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:01:16.820564 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:01:16.825594 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85e315d-9616-49d3-af9a-9af00ad8b615 service neutron] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:16.825958 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85e315d-9616-49d3-af9a-9af00ad8b615 service neutron] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:16.826247 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85e315d-9616-49d3-af9a-9af00ad8b615 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:16.875218 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 880/1723] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jun 23 16:01:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 16:01:16.888176 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85e315d-9616-49d3-af9a-9af00ad8b615 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:01:16.897571 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85e315d-9616-49d3-af9a-9af00ad8b615 service neutron] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:01:16.938742 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 844/1724] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jun 23 16:01:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 16:01:19.554017 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377132c7-ebeb-4b61-91f0-3d94581708a4 None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:19.554902 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377132c7-ebeb-4b61-91f0-3d94581708a4 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:19.555688 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377132c7-ebeb-4b61-91f0-3d94581708a4 None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:19.558873 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 881/1725] 10.222.0.46 () {54 vars in 921 bytes} [Thu Jun 23 16:01:19 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 16:01:20.219368 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85e315d-9616-49d3-af9a-9af00ad8b615 service neutron] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:01:20.220787 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-f85e315d-9616-49d3-af9a-9af00ad8b615 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:01:20.267798 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85e315d-9616-49d3-af9a-9af00ad8b615 service neutron] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:01:20.306985 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7941a6-fd98-4220-95e8-10aa1389c0a7 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:20.306985 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7941a6-fd98-4220-95e8-10aa1389c0a7 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:20.306985 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7941a6-fd98-4220-95e8-10aa1389c0a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:20.319560 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3eebfe0-df50-4013-96af-0a53541a8d98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:01:20.320721 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3eebfe0-df50-4013-96af-0a53541a8d98 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:20.324665 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3eebfe0-df50-4013-96af-0a53541a8d98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:20.326503 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3eebfe0-df50-4013-96af-0a53541a8d98 admin admin] PATH_INFO: `/v3/projects/2975413ebd8643f8aba67386288cb006` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:20.333431 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3eebfe0-df50-4013-96af-0a53541a8d98 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2975413ebd8643f8aba67386288cb006)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:01:20.340108 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3eebfe0-df50-4013-96af-0a53541a8d98 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:01:20.369757 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-9d7941a6-fd98-4220-95e8-10aa1389c0a7 None None] MFA Rules not processed for user `705115d19bab424d894bea8b1b48a7f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:01:20.376035 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-9d7941a6-fd98-4220-95e8-10aa1389c0a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:01:20.629147 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-a3eebfe0-df50-4013-96af-0a53541a8d98 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2975413ebd8643f8aba67386288cb006', 'request_id': 'req-a3eebfe0-df50-4013-96af-0a53541a8d98'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:01:20.658024 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-a3eebfe0-df50-4013-96af-0a53541a8d98 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2975413ebd8643f8aba67386288cb006', 'request_id': 'req-a3eebfe0-df50-4013-96af-0a53541a8d98'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:01:20.676289 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-a3eebfe0-df50-4013-96af-0a53541a8d98 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2975413ebd8643f8aba67386288cb006', 'request_id': 'req-a3eebfe0-df50-4013-96af-0a53541a8d98'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:01:20.720263 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-a3eebfe0-df50-4013-96af-0a53541a8d98 admin admin] The token cache is being invalidate because project 2975413ebd8643f8aba67386288cb006 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80154) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 16:01:20.720606 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-a3eebfe0-df50-4013-96af-0a53541a8d98 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:01:20.722722 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-a3eebfe0-df50-4013-96af-0a53541a8d98 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:01:20.725822 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 845/1726] 10.222.0.46 () {58 vars in 1232 bytes} [Thu Jun 23 16:01:20 2022] DELETE /identity/v3/projects/2975413ebd8643f8aba67386288cb006 => generated 0 bytes in 507 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 16:01:20.730342 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3eebfe0-df50-4013-96af-0a53541a8d98 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:01:20.748672 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-a3eebfe0-df50-4013-96af-0a53541a8d98 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:01:20.757546 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 882/1727] 10.222.0.46 () {58 vars in 1018 bytes} [Thu Jun 23 16:01:20 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 452 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 16:01:20.761105 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7941a6-fd98-4220-95e8-10aa1389c0a7 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:01:20.767144 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-9d7941a6-fd98-4220-95e8-10aa1389c0a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:01:20.768861 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:01:20.792361 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7941a6-fd98-4220-95e8-10aa1389c0a7 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:01:20.935429 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4b3d4f-abf0-46f1-ae61-2344706b3b02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:01:20.939330 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4b3d4f-abf0-46f1-ae61-2344706b3b02 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:20.940102 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4b3d4f-abf0-46f1-ae61-2344706b3b02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:20.943368 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4b3d4f-abf0-46f1-ae61-2344706b3b02 admin admin] PATH_INFO: `/v3/users/8f76af88aa0b40fab5833df2d32b6f54` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:20.961105 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4b3d4f-abf0-46f1-ae61-2344706b3b02 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8f76af88aa0b40fab5833df2d32b6f54)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:01:20.969370 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4b3d4f-abf0-46f1-ae61-2344706b3b02 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:01:20.993500 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3eebfe0-df50-4013-96af-0a53541a8d98 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:01:21.024822 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1eff0f-9c7f-4384-9692-782d2a4315c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:01:21.026016 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1eff0f-9c7f-4384-9692-782d2a4315c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:21.026299 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1eff0f-9c7f-4384-9692-782d2a4315c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:21.026600 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1eff0f-9c7f-4384-9692-782d2a4315c1 admin admin] PATH_INFO: `/v3/projects/e65b4e064fd6448896fb26b9ad2e85ac` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:21.039842 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1eff0f-9c7f-4384-9692-782d2a4315c1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e65b4e064fd6448896fb26b9ad2e85ac)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:01:21.042675 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1eff0f-9c7f-4384-9692-782d2a4315c1 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:01:21.170644 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-ff4b3d4f-abf0-46f1-ae61-2344706b3b02 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8f76af88aa0b40fab5833df2d32b6f54'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:01:21.188100 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-ff4b3d4f-abf0-46f1-ae61-2344706b3b02 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8f76af88aa0b40fab5833df2d32b6f54'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:01:21.190167 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-ff4b3d4f-abf0-46f1-ae61-2344706b3b02 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8f76af88aa0b40fab5833df2d32b6f54'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:01:21.191216 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-ff4b3d4f-abf0-46f1-ae61-2344706b3b02 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8f76af88aa0b40fab5833df2d32b6f54'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:01:21.249728 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-ff4b3d4f-abf0-46f1-ae61-2344706b3b02 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8f76af88aa0b40fab5833df2d32b6f54'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:01:21.295147 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 883/1728] 10.222.0.46 () {58 vars in 1223 bytes} [Thu Jun 23 16:01:20 2022] DELETE /identity/v3/users/8f76af88aa0b40fab5833df2d32b6f54 => generated 0 bytes in 536 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 16:01:21.350286 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-3d1eff0f-9c7f-4384-9692-782d2a4315c1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e65b4e064fd6448896fb26b9ad2e85ac', 'request_id': 'req-3d1eff0f-9c7f-4384-9692-782d2a4315c1'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:01:21.381851 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-3d1eff0f-9c7f-4384-9692-782d2a4315c1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e65b4e064fd6448896fb26b9ad2e85ac', 'request_id': 'req-3d1eff0f-9c7f-4384-9692-782d2a4315c1'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:01:21.408093 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-3d1eff0f-9c7f-4384-9692-782d2a4315c1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e65b4e064fd6448896fb26b9ad2e85ac', 'request_id': 'req-3d1eff0f-9c7f-4384-9692-782d2a4315c1'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:01:21.460832 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-3d1eff0f-9c7f-4384-9692-782d2a4315c1 admin admin] The token cache is being invalidate because project e65b4e064fd6448896fb26b9ad2e85ac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80154) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 16:01:21.461162 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-3d1eff0f-9c7f-4384-9692-782d2a4315c1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:01:21.466192 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-3d1eff0f-9c7f-4384-9692-782d2a4315c1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:01:21.475784 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 846/1729] 10.222.0.46 () {58 vars in 1232 bytes} [Thu Jun 23 16:01:20 2022] DELETE /identity/v3/projects/e65b4e064fd6448896fb26b9ad2e85ac => generated 0 bytes in 747 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 16:01:21.477428 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4b3d4f-abf0-46f1-ae61-2344706b3b02 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:01:21.479900 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-ff4b3d4f-abf0-46f1-ae61-2344706b3b02 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:01:21.543096 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:01:21.760786 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4b3d4f-abf0-46f1-ae61-2344706b3b02 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:01:21.785541 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e583683-dc46-4845-838e-1326464636b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:01:21.788487 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e583683-dc46-4845-838e-1326464636b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:21.788487 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e583683-dc46-4845-838e-1326464636b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:21.788487 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e583683-dc46-4845-838e-1326464636b2 admin admin] PATH_INFO: `/v3/users/67ca75bef7a047e8966f7973a7e0d712` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:21.793543 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e583683-dc46-4845-838e-1326464636b2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=67ca75bef7a047e8966f7973a7e0d712)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:01:21.797584 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e583683-dc46-4845-838e-1326464636b2 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:01:21.857345 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1eff0f-9c7f-4384-9692-782d2a4315c1 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:01:21.859939 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-3d1eff0f-9c7f-4384-9692-782d2a4315c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:01:21.913972 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-6e583683-dc46-4845-838e-1326464636b2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '67ca75bef7a047e8966f7973a7e0d712'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:01:21.925803 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-6e583683-dc46-4845-838e-1326464636b2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '67ca75bef7a047e8966f7973a7e0d712'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:01:21.926771 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-6e583683-dc46-4845-838e-1326464636b2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '67ca75bef7a047e8966f7973a7e0d712'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:01:21.927461 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-6e583683-dc46-4845-838e-1326464636b2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '67ca75bef7a047e8966f7973a7e0d712'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:01:21.952117 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-6e583683-dc46-4845-838e-1326464636b2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '67ca75bef7a047e8966f7973a7e0d712'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:01:21.975049 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 884/1730] 10.222.0.46 () {58 vars in 1223 bytes} [Thu Jun 23 16:01:21 2022] DELETE /identity/v3/users/67ca75bef7a047e8966f7973a7e0d712 => generated 0 bytes in 498 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 16:01:22.180117 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1eff0f-9c7f-4384-9692-782d2a4315c1 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:01:22.271757 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:01:22.321088 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521c844d-2e0c-4eff-9159-6c7e621c5f5b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '705115d19bab424d894bea8b1b48a7f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:01:22.322759 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521c844d-2e0c-4eff-9159-6c7e621c5f5b service nova] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:22.323361 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521c844d-2e0c-4eff-9159-6c7e621c5f5b service nova] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:22.323674 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521c844d-2e0c-4eff-9159-6c7e621c5f5b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:22.326380 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-521c844d-2e0c-4eff-9159-6c7e621c5f5b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:01:22.343827 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521c844d-2e0c-4eff-9159-6c7e621c5f5b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:01:22.346023 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521c844d-2e0c-4eff-9159-6c7e621c5f5b service nova] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:01:22.384547 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 847/1731] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jun 23 16:01:21 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 528 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 16:01:23.760557 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e583683-dc46-4845-838e-1326464636b2 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:01:23.762481 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-6e583683-dc46-4845-838e-1326464636b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:01:23.893779 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521c844d-2e0c-4eff-9159-6c7e621c5f5b service nova] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:01:23.898402 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-521c844d-2e0c-4eff-9159-6c7e621c5f5b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:01:24.007320 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e583683-dc46-4845-838e-1326464636b2 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:01:24.023830 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e930537-c3c5-45df-aafa-6fe0958529a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:01:24.025781 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e930537-c3c5-45df-aafa-6fe0958529a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:24.026069 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e930537-c3c5-45df-aafa-6fe0958529a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:24.026350 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e930537-c3c5-45df-aafa-6fe0958529a3 admin admin] PATH_INFO: `/v3/projects/a3cb13a757a746a2915ca1e8d9ad12b0` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:24.032887 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e930537-c3c5-45df-aafa-6fe0958529a3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a3cb13a757a746a2915ca1e8d9ad12b0)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:01:24.034900 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e930537-c3c5-45df-aafa-6fe0958529a3 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:01:24.079614 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521c844d-2e0c-4eff-9159-6c7e621c5f5b service nova] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:01:24.091803 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb30fb2a-cf9a-46dd-a2ad-e3cf15e77c2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:01:24.092370 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb30fb2a-cf9a-46dd-a2ad-e3cf15e77c2b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:24.093005 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb30fb2a-cf9a-46dd-a2ad-e3cf15e77c2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:24.095586 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb30fb2a-cf9a-46dd-a2ad-e3cf15e77c2b admin admin] PATH_INFO: `/v3/projects/fe4cb80c54f3454a897675c905ef1732` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:24.100419 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb30fb2a-cf9a-46dd-a2ad-e3cf15e77c2b admin admin] RBAC: Authorizing `identity:delete_project(project_id=fe4cb80c54f3454a897675c905ef1732)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:01:24.102405 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb30fb2a-cf9a-46dd-a2ad-e3cf15e77c2b admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:01:24.132161 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:01:24.168088 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-0e930537-c3c5-45df-aafa-6fe0958529a3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a3cb13a757a746a2915ca1e8d9ad12b0', 'request_id': 'req-0e930537-c3c5-45df-aafa-6fe0958529a3'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:01:24.188483 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-0e930537-c3c5-45df-aafa-6fe0958529a3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a3cb13a757a746a2915ca1e8d9ad12b0', 'request_id': 'req-0e930537-c3c5-45df-aafa-6fe0958529a3'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:01:24.208556 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-0e930537-c3c5-45df-aafa-6fe0958529a3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a3cb13a757a746a2915ca1e8d9ad12b0', 'request_id': 'req-0e930537-c3c5-45df-aafa-6fe0958529a3'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:01:24.233564 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-fb30fb2a-cf9a-46dd-a2ad-e3cf15e77c2b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fe4cb80c54f3454a897675c905ef1732', 'request_id': 'req-fb30fb2a-cf9a-46dd-a2ad-e3cf15e77c2b'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:01:24.242319 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-0e930537-c3c5-45df-aafa-6fe0958529a3 admin admin] The token cache is being invalidate because project a3cb13a757a746a2915ca1e8d9ad12b0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80153) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 16:01:24.242651 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-0e930537-c3c5-45df-aafa-6fe0958529a3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:01:24.243167 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-fb30fb2a-cf9a-46dd-a2ad-e3cf15e77c2b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fe4cb80c54f3454a897675c905ef1732', 'request_id': 'req-fb30fb2a-cf9a-46dd-a2ad-e3cf15e77c2b'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:01:24.245950 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-0e930537-c3c5-45df-aafa-6fe0958529a3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:01:24.251214 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 885/1732] 10.222.0.46 () {58 vars in 1232 bytes} [Thu Jun 23 16:01:23 2022] DELETE /identity/v3/projects/a3cb13a757a746a2915ca1e8d9ad12b0 => generated 0 bytes in 488 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 16:01:24.255235 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55413b4a-2062-4e72-823a-832e12022a69 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:24.255542 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55413b4a-2062-4e72-823a-832e12022a69 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:24.255827 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55413b4a-2062-4e72-823a-832e12022a69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:24.255931 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-fb30fb2a-cf9a-46dd-a2ad-e3cf15e77c2b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fe4cb80c54f3454a897675c905ef1732', 'request_id': 'req-fb30fb2a-cf9a-46dd-a2ad-e3cf15e77c2b'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:01:24.283628 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-55413b4a-2062-4e72-823a-832e12022a69 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:01:24.284949 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-55413b4a-2062-4e72-823a-832e12022a69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:01:24.287938 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-fb30fb2a-cf9a-46dd-a2ad-e3cf15e77c2b admin admin] The token cache is being invalidate because project fe4cb80c54f3454a897675c905ef1732 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80154) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 16:01:24.288240 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-fb30fb2a-cf9a-46dd-a2ad-e3cf15e77c2b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:01:24.289065 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-fb30fb2a-cf9a-46dd-a2ad-e3cf15e77c2b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:01:24.291039 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 848/1733] 10.222.0.46 () {58 vars in 1232 bytes} [Thu Jun 23 16:01:23 2022] DELETE /identity/v3/projects/fe4cb80c54f3454a897675c905ef1732 => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 16:01:24.295911 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a962d6af-c322-4ec7-a805-c58a2ed726ac None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:24.296325 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a962d6af-c322-4ec7-a805-c58a2ed726ac None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:24.296713 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a962d6af-c322-4ec7-a805-c58a2ed726ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:24.322463 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-a962d6af-c322-4ec7-a805-c58a2ed726ac None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:01:24.324206 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-a962d6af-c322-4ec7-a805-c58a2ed726ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:01:24.386733 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 886/1734] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:01:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 16:01:24.398775 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6f2b2d-0a04-44ab-badf-a13c6ff2ddba None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:24.401914 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6f2b2d-0a04-44ab-badf-a13c6ff2ddba None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:24.402566 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6f2b2d-0a04-44ab-badf-a13c6ff2ddba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:24.421675 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:01:24.433134 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-9e6f2b2d-0a04-44ab-badf-a13c6ff2ddba None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:01:24.434489 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-9e6f2b2d-0a04-44ab-badf-a13c6ff2ddba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:01:24.440208 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 849/1735] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:01:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 16:01:24.441548 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a962d6af-c322-4ec7-a805-c58a2ed726ac None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:01:24.443306 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-a962d6af-c322-4ec7-a805-c58a2ed726ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:01:24.444405 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 887/1736] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:01:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 16:01:24.450238 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb6dabd-be3c-465e-b9ca-4d2c0e139f34 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:24.451246 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb6dabd-be3c-465e-b9ca-4d2c0e139f34 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:24.451384 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb6dabd-be3c-465e-b9ca-4d2c0e139f34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:24.461409 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a962d6af-c322-4ec7-a805-c58a2ed726ac None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:01:24.474164 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528f6452-9625-4b73-8ab0-4f2edfbc6df0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:01:24.476527 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528f6452-9625-4b73-8ab0-4f2edfbc6df0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:24.476527 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528f6452-9625-4b73-8ab0-4f2edfbc6df0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:24.476527 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528f6452-9625-4b73-8ab0-4f2edfbc6df0 admin admin] PATH_INFO: `/v3/projects/8ec6f914ff6d4ccda98d260d15c78e60` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:24.481442 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528f6452-9625-4b73-8ab0-4f2edfbc6df0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8ec6f914ff6d4ccda98d260d15c78e60)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:01:24.487294 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528f6452-9625-4b73-8ab0-4f2edfbc6df0 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:01:24.527838 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-fcb6dabd-be3c-465e-b9ca-4d2c0e139f34 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:01:24.530373 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-fcb6dabd-be3c-465e-b9ca-4d2c0e139f34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:01:24.544986 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 888/1737] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:01:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 16:01:24.544986 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb6dabd-be3c-465e-b9ca-4d2c0e139f34 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:01:24.555085 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb6dabd-be3c-465e-b9ca-4d2c0e139f34 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:01:24.569274 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-528f6452-9625-4b73-8ab0-4f2edfbc6df0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8ec6f914ff6d4ccda98d260d15c78e60', 'request_id': 'req-528f6452-9625-4b73-8ab0-4f2edfbc6df0'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:01:24.584567 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-528f6452-9625-4b73-8ab0-4f2edfbc6df0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8ec6f914ff6d4ccda98d260d15c78e60', 'request_id': 'req-528f6452-9625-4b73-8ab0-4f2edfbc6df0'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:01:24.599676 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-528f6452-9625-4b73-8ab0-4f2edfbc6df0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8ec6f914ff6d4ccda98d260d15c78e60', 'request_id': 'req-528f6452-9625-4b73-8ab0-4f2edfbc6df0'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:01:24.630647 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-528f6452-9625-4b73-8ab0-4f2edfbc6df0 admin admin] The token cache is being invalidate because project 8ec6f914ff6d4ccda98d260d15c78e60 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80154) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 16:01:24.631255 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-528f6452-9625-4b73-8ab0-4f2edfbc6df0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:01:24.632463 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-528f6452-9625-4b73-8ab0-4f2edfbc6df0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:01:24.640110 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 850/1738] 10.222.0.46 () {58 vars in 1232 bytes} [Thu Jun 23 16:01:24 2022] DELETE /identity/v3/projects/8ec6f914ff6d4ccda98d260d15c78e60 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 16:01:24.640110 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528f6452-9625-4b73-8ab0-4f2edfbc6df0 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:01:24.641750 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-528f6452-9625-4b73-8ab0-4f2edfbc6df0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:01:24.650245 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1b1229-caad-433c-8f1a-f94659624564 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:01:24.651955 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1b1229-caad-433c-8f1a-f94659624564 admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:24.652240 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1b1229-caad-433c-8f1a-f94659624564 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:24.652517 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1b1229-caad-433c-8f1a-f94659624564 admin admin] PATH_INFO: `/v3/domains` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:24.653258 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1b1229-caad-433c-8f1a-f94659624564 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 16:01:24.653565 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1b1229-caad-433c-8f1a-f94659624564 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:01:24.658574 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1b1229-caad-433c-8f1a-f94659624564 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:01:24.673377 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528f6452-9625-4b73-8ab0-4f2edfbc6df0 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:01:24.682608 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 889/1739] 10.222.0.46 () {58 vars in 1152 bytes} [Thu Jun 23 16:01:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 16:01:24.684175 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd9cae4-bd5b-499a-8605-af99242764c2 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:24.684471 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd9cae4-bd5b-499a-8605-af99242764c2 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:24.684751 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd9cae4-bd5b-499a-8605-af99242764c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:24.699953 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d75e85a-5709-4210-9166-023d8fde0559 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:01:24.701027 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d75e85a-5709-4210-9166-023d8fde0559 admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:24.701716 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d75e85a-5709-4210-9166-023d8fde0559 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:24.702212 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d75e85a-5709-4210-9166-023d8fde0559 admin admin] PATH_INFO: `/v3/domains` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:24.703264 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d75e85a-5709-4210-9166-023d8fde0559 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 16:01:24.704906 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d75e85a-5709-4210-9166-023d8fde0559 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:01:24.709129 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-8cd9cae4-bd5b-499a-8605-af99242764c2 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:01:24.710330 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d75e85a-5709-4210-9166-023d8fde0559 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:01:24.711582 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-8cd9cae4-bd5b-499a-8605-af99242764c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:01:24.721994 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 890/1740] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:01:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 16:01:24.722573 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd9cae4-bd5b-499a-8605-af99242764c2 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:01:24.725770 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-8cd9cae4-bd5b-499a-8605-af99242764c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:01:24.744993 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd9cae4-bd5b-499a-8605-af99242764c2 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:01:24.751819 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b73965-ff9d-4ca2-8d6b-690b84452ef5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:01:24.752673 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b73965-ff9d-4ca2-8d6b-690b84452ef5 admin admin] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:24.753105 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b73965-ff9d-4ca2-8d6b-690b84452ef5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:24.753277 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b73965-ff9d-4ca2-8d6b-690b84452ef5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:24.754057 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b73965-ff9d-4ca2-8d6b-690b84452ef5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:01:24.756185 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 851/1741] 10.222.0.46 () {58 vars in 1152 bytes} [Thu Jun 23 16:01:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 16:01:24.756392 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b73965-ff9d-4ca2-8d6b-690b84452ef5 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:01:24.761030 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53772f6f-4702-4a36-8737-b64aecd88ae5 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:24.761305 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53772f6f-4702-4a36-8737-b64aecd88ae5 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:24.761742 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53772f6f-4702-4a36-8737-b64aecd88ae5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:24.805724 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 891/1742] 10.222.0.46 () {60 vars in 1152 bytes} [Thu Jun 23 16:01:24 2022] POST /identity/v3/projects => generated 357 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 16:01:24.805724 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b73965-ff9d-4ca2-8d6b-690b84452ef5 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:01:24.819771 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-53772f6f-4702-4a36-8737-b64aecd88ae5 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:01:24.821982 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-53772f6f-4702-4a36-8737-b64aecd88ae5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:01:24.835332 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:01:24.918325 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b73965-ff9d-4ca2-8d6b-690b84452ef5 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:01:24.930273 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7713bd-71a6-4b14-bbbe-296946ee9646 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:01:24.930273 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7713bd-71a6-4b14-bbbe-296946ee9646 admin admin] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:24.930273 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7713bd-71a6-4b14-bbbe-296946ee9646 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:24.930818 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7713bd-71a6-4b14-bbbe-296946ee9646 admin admin] PATH_INFO: `/v3/projects` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:24.933964 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7713bd-71a6-4b14-bbbe-296946ee9646 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:01:24.936481 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7713bd-71a6-4b14-bbbe-296946ee9646 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:01:24.999801 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 852/1743] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:01:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 16:01:25.001534 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53772f6f-4702-4a36-8737-b64aecd88ae5 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:01:25.003075 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 892/1744] 10.222.0.46 () {60 vars in 1152 bytes} [Thu Jun 23 16:01:24 2022] POST /identity/v3/projects => generated 405 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 16:01:25.004781 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7713bd-71a6-4b14-bbbe-296946ee9646 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:01:25.029897 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53772f6f-4702-4a36-8737-b64aecd88ae5 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:01:25.172304 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64c9915-11d5-484a-8519-af67ef320ab6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:01:25.173012 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64c9915-11d5-484a-8519-af67ef320ab6 admin admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:25.173299 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64c9915-11d5-484a-8519-af67ef320ab6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:25.173582 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64c9915-11d5-484a-8519-af67ef320ab6 admin admin] PATH_INFO: `/v3/users` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:25.175114 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64c9915-11d5-484a-8519-af67ef320ab6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:01:25.177387 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64c9915-11d5-484a-8519-af67ef320ab6 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:01:25.193386 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7713bd-71a6-4b14-bbbe-296946ee9646 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:01:25.214231 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fc65a1-df03-43fd-8dbb-2c5b6324595d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:01:25.217535 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fc65a1-df03-43fd-8dbb-2c5b6324595d admin admin] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:25.217936 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fc65a1-df03-43fd-8dbb-2c5b6324595d admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:25.218276 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fc65a1-df03-43fd-8dbb-2c5b6324595d admin admin] PATH_INFO: `/v3/users` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:25.219146 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:01:25.219278 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fc65a1-df03-43fd-8dbb-2c5b6324595d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:01:25.222395 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fc65a1-df03-43fd-8dbb-2c5b6324595d admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:01:25.235896 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 853/1745] 10.222.0.46 () {60 vars in 1143 bytes} [Thu Jun 23 16:01:25 2022] POST /identity/v3/users => generated 279 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 16:01:25.237182 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64c9915-11d5-484a-8519-af67ef320ab6 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:01:25.265016 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64c9915-11d5-484a-8519-af67ef320ab6 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:01:25.282412 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 893/1746] 10.222.0.46 () {60 vars in 1143 bytes} [Thu Jun 23 16:01:25 2022] POST /identity/v3/users => generated 303 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 16:01:25.284309 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fc65a1-df03-43fd-8dbb-2c5b6324595d admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:01:25.285966 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf6b87f-08ba-4d32-b949-e50d1c440a8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:01:25.286674 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf6b87f-08ba-4d32-b949-e50d1c440a8a admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:25.287168 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf6b87f-08ba-4d32-b949-e50d1c440a8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:25.287459 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf6b87f-08ba-4d32-b949-e50d1c440a8a admin admin] PATH_INFO: `/v3/domains` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:25.288223 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf6b87f-08ba-4d32-b949-e50d1c440a8a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 16:01:25.288543 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf6b87f-08ba-4d32-b949-e50d1c440a8a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:01:25.290964 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf6b87f-08ba-4d32-b949-e50d1c440a8a admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:01:25.304489 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fc65a1-df03-43fd-8dbb-2c5b6324595d admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:01:25.327711 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd77d83-6e39-4526-a0f5-dd6833ee95e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:01:25.328831 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 854/1747] 10.222.0.46 () {58 vars in 1152 bytes} [Thu Jun 23 16:01:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 16:01:25.329110 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd77d83-6e39-4526-a0f5-dd6833ee95e6 admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:25.329519 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd77d83-6e39-4526-a0f5-dd6833ee95e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:25.329804 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf6b87f-08ba-4d32-b949-e50d1c440a8a admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:01:25.330125 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd77d83-6e39-4526-a0f5-dd6833ee95e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:25.331811 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd77d83-6e39-4526-a0f5-dd6833ee95e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:01:25.333763 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd77d83-6e39-4526-a0f5-dd6833ee95e6 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:01:25.357385 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf6b87f-08ba-4d32-b949-e50d1c440a8a admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:01:25.369483 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 894/1748] 10.222.0.46 () {58 vars in 1121 bytes} [Thu Jun 23 16:01:25 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 16:01:25.371162 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd77d83-6e39-4526-a0f5-dd6833ee95e6 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:01:25.373652 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b709a962-3aed-45f4-9656-29bbcf2f6926 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:01:25.376034 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b709a962-3aed-45f4-9656-29bbcf2f6926 admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:25.377878 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b709a962-3aed-45f4-9656-29bbcf2f6926 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:25.378318 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b709a962-3aed-45f4-9656-29bbcf2f6926 admin admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:25.380534 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b709a962-3aed-45f4-9656-29bbcf2f6926 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:01:25.387660 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b709a962-3aed-45f4-9656-29bbcf2f6926 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:01:25.394712 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd77d83-6e39-4526-a0f5-dd6833ee95e6 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:01:25.408591 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5368d4-fe87-4742-86c2-1b4288f8d620 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:01:25.409783 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5368d4-fe87-4742-86c2-1b4288f8d620 admin admin] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:25.410227 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5368d4-fe87-4742-86c2-1b4288f8d620 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:25.410774 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5368d4-fe87-4742-86c2-1b4288f8d620 admin admin] PATH_INFO: `/v3/projects` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:25.411898 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5368d4-fe87-4742-86c2-1b4288f8d620 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:01:25.415223 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 855/1749] 10.222.0.46 () {58 vars in 1121 bytes} [Thu Jun 23 16:01:25 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 16:01:25.416992 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5368d4-fe87-4742-86c2-1b4288f8d620 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:01:25.419142 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b709a962-3aed-45f4-9656-29bbcf2f6926 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:01:25.445056 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b709a962-3aed-45f4-9656-29bbcf2f6926 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:01:25.457333 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2453fa54-ee47-47c2-a724-d3c709aec839 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:01:25.458067 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2453fa54-ee47-47c2-a724-d3c709aec839 admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:25.458362 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2453fa54-ee47-47c2-a724-d3c709aec839 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:25.460418 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2453fa54-ee47-47c2-a724-d3c709aec839 admin admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:25.461535 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2453fa54-ee47-47c2-a724-d3c709aec839 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:01:25.468015 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2453fa54-ee47-47c2-a724-d3c709aec839 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:01:25.482185 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 895/1750] 10.222.0.46 () {60 vars in 1152 bytes} [Thu Jun 23 16:01:25 2022] POST /identity/v3/projects => generated 385 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 16:01:25.483322 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5368d4-fe87-4742-86c2-1b4288f8d620 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:01:25.497527 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 856/1751] 10.222.0.46 () {58 vars in 1121 bytes} [Thu Jun 23 16:01:25 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 16:01:25.498414 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2453fa54-ee47-47c2-a724-d3c709aec839 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:01:25.523664 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2453fa54-ee47-47c2-a724-d3c709aec839 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:01:25.603562 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:01:25.623728 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5368d4-fe87-4742-86c2-1b4288f8d620 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:01:25.630135 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14d9dcc-d4ae-4849-8672-d22b56ff776a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:01:25.630956 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14d9dcc-d4ae-4849-8672-d22b56ff776a admin admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:25.631442 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14d9dcc-d4ae-4849-8672-d22b56ff776a admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:25.631675 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14d9dcc-d4ae-4849-8672-d22b56ff776a admin admin] PATH_INFO: `/v3/users` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:25.632440 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14d9dcc-d4ae-4849-8672-d22b56ff776a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:01:25.633779 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93dc30d8-d6ac-434c-aafc-8d43015e3ce2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:01:25.635344 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14d9dcc-d4ae-4849-8672-d22b56ff776a admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:01:25.635628 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dc30d8-d6ac-434c-aafc-8d43015e3ce2 admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:25.636524 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dc30d8-d6ac-434c-aafc-8d43015e3ce2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:25.637004 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dc30d8-d6ac-434c-aafc-8d43015e3ce2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:25.637939 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93dc30d8-d6ac-434c-aafc-8d43015e3ce2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:01:25.640366 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93dc30d8-d6ac-434c-aafc-8d43015e3ce2 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:01:25.670382 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 857/1752] 10.222.0.46 () {60 vars in 1143 bytes} [Thu Jun 23 16:01:25 2022] POST /identity/v3/users => generated 293 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 16:01:25.670382 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14d9dcc-d4ae-4849-8672-d22b56ff776a admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:01:25.677639 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 896/1753] 10.222.0.46 () {58 vars in 1121 bytes} [Thu Jun 23 16:01:25 2022] GET /identity/v3/roles => generated 1420 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 16:01:25.677639 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93dc30d8-d6ac-434c-aafc-8d43015e3ce2 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:01:25.680611 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14d9dcc-d4ae-4849-8672-d22b56ff776a admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:01:25.686545 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef657ea-a3b9-43c0-a544-0937bfea8dda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:01:25.687824 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef657ea-a3b9-43c0-a544-0937bfea8dda admin admin] REQUEST_METHOD: `PUT` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:25.688684 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef657ea-a3b9-43c0-a544-0937bfea8dda admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:25.688684 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef657ea-a3b9-43c0-a544-0937bfea8dda admin admin] PATH_INFO: `/v3/projects/ad22a947e7cc4aa8857a06f120c0354e/users/0f1e3a6043ac4b96b5ee27dc88d7f1e1/roles/e2a5bbf3200a4d4091d72bc7ff78c34d` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:25.704021 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93dc30d8-d6ac-434c-aafc-8d43015e3ce2 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:01:25.713254 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a297a0-59cc-4dff-bb6b-d74e7ed26ed6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:01:25.714626 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a297a0-59cc-4dff-bb6b-d74e7ed26ed6 admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:25.715169 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a297a0-59cc-4dff-bb6b-d74e7ed26ed6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:25.715608 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a297a0-59cc-4dff-bb6b-d74e7ed26ed6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:25.716462 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef657ea-a3b9-43c0-a544-0937bfea8dda admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad22a947e7cc4aa8857a06f120c0354e, user_id=0f1e3a6043ac4b96b5ee27dc88d7f1e1, role_id=e2a5bbf3200a4d4091d72bc7ff78c34d)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:01:25.716748 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a297a0-59cc-4dff-bb6b-d74e7ed26ed6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:01:25.718634 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a297a0-59cc-4dff-bb6b-d74e7ed26ed6 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:01:25.722047 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef657ea-a3b9-43c0-a544-0937bfea8dda admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:01:25.737970 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 897/1754] 10.222.0.46 () {58 vars in 1121 bytes} [Thu Jun 23 16:01:25 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 16:01:25.739637 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a297a0-59cc-4dff-bb6b-d74e7ed26ed6 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:01:25.748233 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 858/1755] 10.222.0.46 () {60 vars in 1482 bytes} [Thu Jun 23 16:01:25 2022] PUT /identity/v3/projects/ad22a947e7cc4aa8857a06f120c0354e/users/0f1e3a6043ac4b96b5ee27dc88d7f1e1/roles/e2a5bbf3200a4d4091d72bc7ff78c34d => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 16:01:25.752395 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a297a0-59cc-4dff-bb6b-d74e7ed26ed6 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:01:25.759758 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef657ea-a3b9-43c0-a544-0937bfea8dda admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:01:25.898118 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff83716-bd2f-4d23-83f6-ed5f0b14b2b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:01:25.899286 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff83716-bd2f-4d23-83f6-ed5f0b14b2b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:25.899709 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff83716-bd2f-4d23-83f6-ed5f0b14b2b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:25.900089 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff83716-bd2f-4d23-83f6-ed5f0b14b2b9 admin admin] PATH_INFO: `/v3/projects/8c0ad2168cdb4120b8a5a8862fda01d3/users/5d8a3488ac9e4c06a7b5f8cba66c0e0d/roles/e2a5bbf3200a4d4091d72bc7ff78c34d` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:25.933182 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef657ea-a3b9-43c0-a544-0937bfea8dda admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:01:25.943564 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d7fccd-675a-467b-a88a-10cdec5acac2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:01:25.950807 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff83716-bd2f-4d23-83f6-ed5f0b14b2b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c0ad2168cdb4120b8a5a8862fda01d3, user_id=5d8a3488ac9e4c06a7b5f8cba66c0e0d, role_id=e2a5bbf3200a4d4091d72bc7ff78c34d)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:01:25.953763 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff83716-bd2f-4d23-83f6-ed5f0b14b2b9 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:01:25.958530 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d7fccd-675a-467b-a88a-10cdec5acac2 admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:25.959263 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d7fccd-675a-467b-a88a-10cdec5acac2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:25.959772 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d7fccd-675a-467b-a88a-10cdec5acac2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:25.961749 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d7fccd-675a-467b-a88a-10cdec5acac2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:01:25.964534 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d7fccd-675a-467b-a88a-10cdec5acac2 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:01:25.981446 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 898/1756] 10.222.0.46 () {60 vars in 1482 bytes} [Thu Jun 23 16:01:25 2022] PUT /identity/v3/projects/8c0ad2168cdb4120b8a5a8862fda01d3/users/5d8a3488ac9e4c06a7b5f8cba66c0e0d/roles/e2a5bbf3200a4d4091d72bc7ff78c34d => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 16:01:25.988635 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35a6948-a3df-468a-bca5-82b908ad13ca None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:25.988635 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35a6948-a3df-468a-bca5-82b908ad13ca None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:25.988893 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35a6948-a3df-468a-bca5-82b908ad13ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:26.003351 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 859/1757] 10.222.0.46 () {58 vars in 1121 bytes} [Thu Jun 23 16:01:25 2022] GET /identity/v3/roles => generated 1420 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 16:01:26.003351 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36163a6d-f37c-4356-ac4f-28d24b6330a9 None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:26.003351 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36163a6d-f37c-4356-ac4f-28d24b6330a9 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:26.004226 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36163a6d-f37c-4356-ac4f-28d24b6330a9 None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:26.006418 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 860/1758] 10.222.0.46 () {54 vars in 921 bytes} [Thu Jun 23 16:01:26 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 16:01:26.010220 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab283e04-d159-4b98-853a-77bcf63f91b5 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:26.010682 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab283e04-d159-4b98-853a-77bcf63f91b5 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:26.011285 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab283e04-d159-4b98-853a-77bcf63f91b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:26.038179 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-ab283e04-d159-4b98-853a-77bcf63f91b5 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:01:26.039683 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-ab283e04-d159-4b98-853a-77bcf63f91b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:01:26.051886 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-a35a6948-a3df-468a-bca5-82b908ad13ca None None] MFA Rules not processed for user `0f1e3a6043ac4b96b5ee27dc88d7f1e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:01:26.053260 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-a35a6948-a3df-468a-bca5-82b908ad13ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:01:26.079185 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:01:26.171306 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 861/1759] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:01:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 16:01:26.173311 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab283e04-d159-4b98-853a-77bcf63f91b5 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:01:26.205662 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab283e04-d159-4b98-853a-77bcf63f91b5 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:01:26.206190 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 899/1760] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:01:25 2022] POST /identity/v3/auth/tokens => generated 3224 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 16:01:26.208988 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2675cd52-a315-4288-aa4f-88c19bd3eede None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:26.208988 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2675cd52-a315-4288-aa4f-88c19bd3eede None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:26.208988 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2675cd52-a315-4288-aa4f-88c19bd3eede None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:26.231009 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcad2ed7-199c-4fe1-a471-4f5d13675a2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:01:26.232326 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcad2ed7-199c-4fe1-a471-4f5d13675a2d admin admin] REQUEST_METHOD: `PUT` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:26.235823 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcad2ed7-199c-4fe1-a471-4f5d13675a2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:26.235823 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcad2ed7-199c-4fe1-a471-4f5d13675a2d admin admin] PATH_INFO: `/v3/projects/dc9df897282446b5b6c4c1c346d1da7f/users/1ebb84bf1a924a409a371f5cb9a9460b/roles/e2a5bbf3200a4d4091d72bc7ff78c34d` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:26.244336 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-2675cd52-a315-4288-aa4f-88c19bd3eede None None] MFA Rules not processed for user `705115d19bab424d894bea8b1b48a7f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:01:26.247033 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-2675cd52-a315-4288-aa4f-88c19bd3eede None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:01:26.261475 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcad2ed7-199c-4fe1-a471-4f5d13675a2d admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc9df897282446b5b6c4c1c346d1da7f, user_id=1ebb84bf1a924a409a371f5cb9a9460b, role_id=e2a5bbf3200a4d4091d72bc7ff78c34d)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:01:26.265550 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcad2ed7-199c-4fe1-a471-4f5d13675a2d admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:01:26.302049 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 862/1761] 10.222.0.46 () {60 vars in 1482 bytes} [Thu Jun 23 16:01:26 2022] PUT /identity/v3/projects/dc9df897282446b5b6c4c1c346d1da7f/users/1ebb84bf1a924a409a371f5cb9a9460b/roles/e2a5bbf3200a4d4091d72bc7ff78c34d => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 16:01:26.303540 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcad2ed7-199c-4fe1-a471-4f5d13675a2d admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:01:26.306877 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-dcad2ed7-199c-4fe1-a471-4f5d13675a2d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:01:26.385834 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:01:26.480120 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 900/1762] 10.222.0.46 () {58 vars in 1018 bytes} [Thu Jun 23 16:01:26 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 273 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 16:01:26.482719 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2675cd52-a315-4288-aa4f-88c19bd3eede None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:01:26.518561 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcad2ed7-199c-4fe1-a471-4f5d13675a2d admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:01:26.520009 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2675cd52-a315-4288-aa4f-88c19bd3eede None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:01:26.535005 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0247454e-60bd-462d-861a-ea9ccbf86f6c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51784064951a4619ae09b9167fb0fd0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:01:26.536562 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0247454e-60bd-462d-861a-ea9ccbf86f6c service neutron] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:26.536853 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0247454e-60bd-462d-861a-ea9ccbf86f6c service neutron] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:26.537139 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0247454e-60bd-462d-861a-ea9ccbf86f6c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:26.683822 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9df147d-1d4c-49c1-94d7-043676fb23b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:01:26.684800 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9df147d-1d4c-49c1-94d7-043676fb23b7 admin admin] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:26.685236 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9df147d-1d4c-49c1-94d7-043676fb23b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:26.685651 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9df147d-1d4c-49c1-94d7-043676fb23b7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:26.686639 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9df147d-1d4c-49c1-94d7-043676fb23b7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:01:26.688990 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9df147d-1d4c-49c1-94d7-043676fb23b7 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:01:26.737443 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0247454e-60bd-462d-861a-ea9ccbf86f6c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:01:26.739149 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0247454e-60bd-462d-861a-ea9ccbf86f6c service neutron] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:01:26.761185 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 863/1763] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jun 23 16:01:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 458 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 16:01:26.762855 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700db8ec-31ec-447b-89a1-864015b82cdd None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:26.763287 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700db8ec-31ec-447b-89a1-864015b82cdd None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:26.763412 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700db8ec-31ec-447b-89a1-864015b82cdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:26.778735 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 901/1764] 10.222.0.46 () {60 vars in 1152 bytes} [Thu Jun 23 16:01:26 2022] POST /identity/v3/projects => generated 357 bytes in 298 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 16:01:26.780043 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9df147d-1d4c-49c1-94d7-043676fb23b7 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:01:26.856198 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-700db8ec-31ec-447b-89a1-864015b82cdd None None] MFA Rules not processed for user `1ebb84bf1a924a409a371f5cb9a9460b`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:01:26.861551 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-700db8ec-31ec-447b-89a1-864015b82cdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:01:26.947461 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9df147d-1d4c-49c1-94d7-043676fb23b7 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:01:26.948788 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:01:26.959390 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3944cfc6-af32-474c-bb1d-c29187dc6874 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:01:26.960092 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3944cfc6-af32-474c-bb1d-c29187dc6874 admin admin] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:26.960381 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3944cfc6-af32-474c-bb1d-c29187dc6874 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:26.960665 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3944cfc6-af32-474c-bb1d-c29187dc6874 admin admin] PATH_INFO: `/v3/users` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:26.967288 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3944cfc6-af32-474c-bb1d-c29187dc6874 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:01:26.969485 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3944cfc6-af32-474c-bb1d-c29187dc6874 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:01:26.995349 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 864/1765] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:01:26 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 16:01:26.996299 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700db8ec-31ec-447b-89a1-864015b82cdd None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:01:26.998185 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-700db8ec-31ec-447b-89a1-864015b82cdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:01:27.019145 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 902/1766] 10.222.0.46 () {60 vars in 1143 bytes} [Thu Jun 23 16:01:26 2022] POST /identity/v3/users => generated 285 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 16:01:27.020345 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3944cfc6-af32-474c-bb1d-c29187dc6874 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:01:27.047904 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3944cfc6-af32-474c-bb1d-c29187dc6874 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:01:27.059389 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6360cd-16cb-4b36-af5d-05baf71d044e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:01:27.060786 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6360cd-16cb-4b36-af5d-05baf71d044e admin admin] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:27.061176 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6360cd-16cb-4b36-af5d-05baf71d044e admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:27.061628 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6360cd-16cb-4b36-af5d-05baf71d044e admin admin] PATH_INFO: `/v3/projects` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:27.063793 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6360cd-16cb-4b36-af5d-05baf71d044e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:01:27.069197 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6360cd-16cb-4b36-af5d-05baf71d044e admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:01:27.138846 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 903/1767] 10.222.0.46 () {60 vars in 1152 bytes} [Thu Jun 23 16:01:27 2022] POST /identity/v3/projects => generated 387 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 16:01:27.140134 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6360cd-16cb-4b36-af5d-05baf71d044e admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:01:27.160357 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700db8ec-31ec-447b-89a1-864015b82cdd None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:01:27.237023 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:01:27.319019 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c11aaa5-890c-400c-a4c2-9a1d5d0f5891 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '705115d19bab424d894bea8b1b48a7f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:01:27.320155 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c11aaa5-890c-400c-a4c2-9a1d5d0f5891 service nova] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:27.320749 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c11aaa5-890c-400c-a4c2-9a1d5d0f5891 service nova] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:27.321699 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c11aaa5-890c-400c-a4c2-9a1d5d0f5891 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:27.321856 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6360cd-16cb-4b36-af5d-05baf71d044e admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:01:27.328428 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272d1266-490d-4831-96cc-422d47dda2c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:01:27.330452 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272d1266-490d-4831-96cc-422d47dda2c1 admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:27.330793 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272d1266-490d-4831-96cc-422d47dda2c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:27.331038 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272d1266-490d-4831-96cc-422d47dda2c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:27.331999 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272d1266-490d-4831-96cc-422d47dda2c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:01:27.335032 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272d1266-490d-4831-96cc-422d47dda2c1 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:01:27.341112 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c11aaa5-890c-400c-a4c2-9a1d5d0f5891 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:01:27.343169 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c11aaa5-890c-400c-a4c2-9a1d5d0f5891 service nova] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:01:27.351164 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 904/1768] 10.222.0.46 () {58 vars in 1121 bytes} [Thu Jun 23 16:01:27 2022] GET /identity/v3/roles => generated 1420 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 16:01:27.353681 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272d1266-490d-4831-96cc-422d47dda2c1 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:01:27.371845 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 865/1769] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jun 23 16:01:26 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 16:01:27.372872 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c11aaa5-890c-400c-a4c2-9a1d5d0f5891 service nova] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:01:27.401996 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c11aaa5-890c-400c-a4c2-9a1d5d0f5891 service nova] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:01:27.407995 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272d1266-490d-4831-96cc-422d47dda2c1 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:01:27.414848 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf1da8e-20e1-4f0a-ad7a-ec26ec0b79c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:01:27.416363 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf1da8e-20e1-4f0a-ad7a-ec26ec0b79c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:27.416835 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf1da8e-20e1-4f0a-ad7a-ec26ec0b79c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:27.417285 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf1da8e-20e1-4f0a-ad7a-ec26ec0b79c9 admin admin] PATH_INFO: `/v3/projects/a2c187698a23468cafa0271182693783/users/100dff85187245af9bcb1eb3f1adc67c/roles/812d3c567c52469cb2b35709fbdc857c` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:27.428336 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4a5612-54ba-40cb-8742-67e1f492d3c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:01:27.429540 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4a5612-54ba-40cb-8742-67e1f492d3c9 admin admin] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:27.430103 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4a5612-54ba-40cb-8742-67e1f492d3c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:27.430559 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4a5612-54ba-40cb-8742-67e1f492d3c9 admin admin] PATH_INFO: `/v3/users` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:27.444254 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf1da8e-20e1-4f0a-ad7a-ec26ec0b79c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a2c187698a23468cafa0271182693783, user_id=100dff85187245af9bcb1eb3f1adc67c, role_id=812d3c567c52469cb2b35709fbdc857c)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:01:27.445511 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf1da8e-20e1-4f0a-ad7a-ec26ec0b79c9 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:01:27.455301 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4a5612-54ba-40cb-8742-67e1f492d3c9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:01:27.464272 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4a5612-54ba-40cb-8742-67e1f492d3c9 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:01:27.469699 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 866/1770] 10.222.0.46 () {60 vars in 1482 bytes} [Thu Jun 23 16:01:27 2022] PUT /identity/v3/projects/a2c187698a23468cafa0271182693783/users/100dff85187245af9bcb1eb3f1adc67c/roles/812d3c567c52469cb2b35709fbdc857c => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 16:01:27.475358 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf1da8e-20e1-4f0a-ad7a-ec26ec0b79c9 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:01:27.520722 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 905/1771] 10.222.0.46 () {60 vars in 1143 bytes} [Thu Jun 23 16:01:27 2022] POST /identity/v3/users => generated 300 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 16:01:27.534877 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4a5612-54ba-40cb-8742-67e1f492d3c9 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:01:27.568549 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:01:27.570940 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4a5612-54ba-40cb-8742-67e1f492d3c9 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:01:27.602490 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf1da8e-20e1-4f0a-ad7a-ec26ec0b79c9 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:01:27.608179 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4434a58-4198-4baf-9947-af4761ab7c6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:01:27.609180 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4434a58-4198-4baf-9947-af4761ab7c6e admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:27.610100 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4434a58-4198-4baf-9947-af4761ab7c6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:27.610100 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4434a58-4198-4baf-9947-af4761ab7c6e admin admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:27.611193 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4434a58-4198-4baf-9947-af4761ab7c6e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:01:27.613084 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4434a58-4198-4baf-9947-af4761ab7c6e admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:01:27.630763 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 867/1772] 10.222.0.46 () {58 vars in 1121 bytes} [Thu Jun 23 16:01:27 2022] GET /identity/v3/roles => generated 1420 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 16:01:27.636746 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4434a58-4198-4baf-9947-af4761ab7c6e admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:01:27.658767 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4434a58-4198-4baf-9947-af4761ab7c6e admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:01:27.671739 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6ad961-d364-4941-b29f-8c4e1b8e8931 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:01:27.674410 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6ad961-d364-4941-b29f-8c4e1b8e8931 admin admin] REQUEST_METHOD: `PUT` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:27.675018 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6ad961-d364-4941-b29f-8c4e1b8e8931 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:27.676169 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6ad961-d364-4941-b29f-8c4e1b8e8931 admin admin] PATH_INFO: `/v3/projects/a2c187698a23468cafa0271182693783/users/100dff85187245af9bcb1eb3f1adc67c/roles/812d3c567c52469cb2b35709fbdc857c` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:27.683286 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6ad961-d364-4941-b29f-8c4e1b8e8931 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a2c187698a23468cafa0271182693783, user_id=100dff85187245af9bcb1eb3f1adc67c, role_id=812d3c567c52469cb2b35709fbdc857c)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:01:27.691428 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6ad961-d364-4941-b29f-8c4e1b8e8931 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:01:27.714309 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 868/1773] 10.222.0.46 () {60 vars in 1482 bytes} [Thu Jun 23 16:01:27 2022] PUT /identity/v3/projects/a2c187698a23468cafa0271182693783/users/100dff85187245af9bcb1eb3f1adc67c/roles/812d3c567c52469cb2b35709fbdc857c => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 16:01:27.731309 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e84e13-5d67-4d26-8e4f-3b5536c6d461 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:27.731935 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e84e13-5d67-4d26-8e4f-3b5536c6d461 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:27.733139 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e84e13-5d67-4d26-8e4f-3b5536c6d461 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:27.736136 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c60827a-620a-4c7d-9814-c30f38c58ea4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:01:27.738573 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c60827a-620a-4c7d-9814-c30f38c58ea4 admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:27.739168 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c60827a-620a-4c7d-9814-c30f38c58ea4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:27.740533 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c60827a-620a-4c7d-9814-c30f38c58ea4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:27.742182 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c60827a-620a-4c7d-9814-c30f38c58ea4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:01:27.746286 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c60827a-620a-4c7d-9814-c30f38c58ea4 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:01:27.774394 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 906/1774] 10.222.0.46 () {58 vars in 1121 bytes} [Thu Jun 23 16:01:27 2022] GET /identity/v3/roles => generated 1420 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 16:01:27.777923 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-90e84e13-5d67-4d26-8e4f-3b5536c6d461 None None] MFA Rules not processed for user `100dff85187245af9bcb1eb3f1adc67c`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:01:27.779714 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-90e84e13-5d67-4d26-8e4f-3b5536c6d461 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:01:27.780683 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c60827a-620a-4c7d-9814-c30f38c58ea4 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:01:27.804242 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c60827a-620a-4c7d-9814-c30f38c58ea4 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:01:27.876057 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 869/1775] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:01:27 2022] POST /identity/v3/auth/tokens => generated 3291 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 16:01:27.889736 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e84e13-5d67-4d26-8e4f-3b5536c6d461 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:01:27.892043 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-90e84e13-5d67-4d26-8e4f-3b5536c6d461 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:01:27.898847 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bd93db-ea9c-49f2-9227-a791e6444108 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:01:27.901528 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bd93db-ea9c-49f2-9227-a791e6444108 admin admin] REQUEST_METHOD: `PUT` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:27.902147 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bd93db-ea9c-49f2-9227-a791e6444108 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:27.902906 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bd93db-ea9c-49f2-9227-a791e6444108 admin admin] PATH_INFO: `/v3/projects/06c66571995f4f6cb86a7ff0d0b5a08e/users/b30b75450fa341fdb8b507e9baabfb4e/roles/812d3c567c52469cb2b35709fbdc857c` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:27.940699 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02bd93db-ea9c-49f2-9227-a791e6444108 admin admin] RBAC: Authorizing `identity:create_grant(project_id=06c66571995f4f6cb86a7ff0d0b5a08e, user_id=b30b75450fa341fdb8b507e9baabfb4e, role_id=812d3c567c52469cb2b35709fbdc857c)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:01:27.946027 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02bd93db-ea9c-49f2-9227-a791e6444108 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:01:27.969119 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e84e13-5d67-4d26-8e4f-3b5536c6d461 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:01:27.972738 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 907/1776] 10.222.0.46 () {60 vars in 1482 bytes} [Thu Jun 23 16:01:27 2022] PUT /identity/v3/projects/06c66571995f4f6cb86a7ff0d0b5a08e/users/b30b75450fa341fdb8b507e9baabfb4e/roles/812d3c567c52469cb2b35709fbdc857c => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 16:01:27.977255 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bd93db-ea9c-49f2-9227-a791e6444108 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:01:28.059785 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb71fa3f-61a3-45fb-add0-32e824e410fe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce18eaaba3a6477684fff1fd98ec8b51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:01:28.061785 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb71fa3f-61a3-45fb-add0-32e824e410fe service cinder] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:28.062176 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb71fa3f-61a3-45fb-add0-32e824e410fe service cinder] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:28.062862 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb71fa3f-61a3-45fb-add0-32e824e410fe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:28.076169 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:01:28.089099 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:01:28.135477 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bd93db-ea9c-49f2-9227-a791e6444108 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:01:28.144118 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ce1ee0-619c-4302-8498-66f8b91d31f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:01:28.145104 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ce1ee0-619c-4302-8498-66f8b91d31f0 admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:28.145424 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ce1ee0-619c-4302-8498-66f8b91d31f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:28.145708 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ce1ee0-619c-4302-8498-66f8b91d31f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:28.146527 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ce1ee0-619c-4302-8498-66f8b91d31f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:01:28.151242 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ce1ee0-619c-4302-8498-66f8b91d31f0 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:01:28.169662 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 908/1777] 10.222.0.46 () {58 vars in 1121 bytes} [Thu Jun 23 16:01:27 2022] GET /identity/v3/roles => generated 1420 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 16:01:28.174837 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ce1ee0-619c-4302-8498-66f8b91d31f0 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:01:28.176122 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb71fa3f-61a3-45fb-add0-32e824e410fe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:01:28.178125 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb71fa3f-61a3-45fb-add0-32e824e410fe service cinder] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:01:28.187069 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ce1ee0-619c-4302-8498-66f8b91d31f0 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:01:28.192820 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef409a6-4384-49a8-84ce-5adc32c0c348 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:01:28.193682 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 870/1778] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jun 23 16:01:27 2022] GET /identity/v3/auth/tokens => generated 3291 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 16:01:28.193885 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef409a6-4384-49a8-84ce-5adc32c0c348 admin admin] REQUEST_METHOD: `PUT` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:28.194085 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef409a6-4384-49a8-84ce-5adc32c0c348 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:28.194363 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef409a6-4384-49a8-84ce-5adc32c0c348 admin admin] PATH_INFO: `/v3/projects/06c66571995f4f6cb86a7ff0d0b5a08e/users/b30b75450fa341fdb8b507e9baabfb4e/roles/812d3c567c52469cb2b35709fbdc857c` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:28.196912 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef409a6-4384-49a8-84ce-5adc32c0c348 admin admin] RBAC: Authorizing `identity:create_grant(project_id=06c66571995f4f6cb86a7ff0d0b5a08e, user_id=b30b75450fa341fdb8b507e9baabfb4e, role_id=812d3c567c52469cb2b35709fbdc857c)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:01:28.199685 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef409a6-4384-49a8-84ce-5adc32c0c348 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:01:28.219648 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 909/1779] 10.222.0.46 () {60 vars in 1482 bytes} [Thu Jun 23 16:01:28 2022] PUT /identity/v3/projects/06c66571995f4f6cb86a7ff0d0b5a08e/users/b30b75450fa341fdb8b507e9baabfb4e/roles/812d3c567c52469cb2b35709fbdc857c => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 16:01:28.231761 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db527ed4-2e86-4d39-8e77-b9603ae014e6 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:28.232044 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db527ed4-2e86-4d39-8e77-b9603ae014e6 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:28.232692 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db527ed4-2e86-4d39-8e77-b9603ae014e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:28.280585 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-db527ed4-2e86-4d39-8e77-b9603ae014e6 None None] MFA Rules not processed for user `b30b75450fa341fdb8b507e9baabfb4e`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:01:28.282492 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-db527ed4-2e86-4d39-8e77-b9603ae014e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:01:28.425136 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef409a6-4384-49a8-84ce-5adc32c0c348 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:01:28.483231 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 871/1780] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:01:28 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 16:01:28.500055 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db527ed4-2e86-4d39-8e77-b9603ae014e6 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:01:28.503036 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-db527ed4-2e86-4d39-8e77-b9603ae014e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:01:28.542797 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef409a6-4384-49a8-84ce-5adc32c0c348 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:01:28.548898 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a3f435-0013-4193-ad43-5e09b2ccaacd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce18eaaba3a6477684fff1fd98ec8b51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:01:28.550423 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a3f435-0013-4193-ad43-5e09b2ccaacd service cinder] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:28.550990 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a3f435-0013-4193-ad43-5e09b2ccaacd service cinder] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:28.551459 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a3f435-0013-4193-ad43-5e09b2ccaacd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:28.602182 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db527ed4-2e86-4d39-8e77-b9603ae014e6 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:01:28.607463 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbe93a2-b1ea-4a3e-aabe-ca69e8a58db0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce18eaaba3a6477684fff1fd98ec8b51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:01:28.608898 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbe93a2-b1ea-4a3e-aabe-ca69e8a58db0 service cinder] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:28.609243 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbe93a2-b1ea-4a3e-aabe-ca69e8a58db0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:28.609615 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbe93a2-b1ea-4a3e-aabe-ca69e8a58db0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:28.626526 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbe93a2-b1ea-4a3e-aabe-ca69e8a58db0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:01:28.628514 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbe93a2-b1ea-4a3e-aabe-ca69e8a58db0 service cinder] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:01:28.645167 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 872/1781] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jun 23 16:01:28 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 16:01:28.652048 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a62e23-ebb6-45c9-a01b-e90acd32ddbb None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:28.652546 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a62e23-ebb6-45c9-a01b-e90acd32ddbb None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:28.653877 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a62e23-ebb6-45c9-a01b-e90acd32ddbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:28.656707 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a3f435-0013-4193-ad43-5e09b2ccaacd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:01:28.658348 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a3f435-0013-4193-ad43-5e09b2ccaacd service cinder] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:01:28.669848 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 910/1782] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jun 23 16:01:28 2022] GET /identity/v3/auth/tokens => generated 3224 bytes in 246 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 16:01:28.753116 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a3f435-0013-4193-ad43-5e09b2ccaacd service cinder] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:01:28.865679 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-53a62e23-ebb6-45c9-a01b-e90acd32ddbb None None] MFA Rules not processed for user `5d8a3488ac9e4c06a7b5f8cba66c0e0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:01:28.868805 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-53a62e23-ebb6-45c9-a01b-e90acd32ddbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:01:28.887939 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a3f435-0013-4193-ad43-5e09b2ccaacd service cinder] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:01:28.895416 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d7bde6-77e4-4620-a988-c78f7715227a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:01:28.896758 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d7bde6-77e4-4620-a988-c78f7715227a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:28.897053 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d7bde6-77e4-4620-a988-c78f7715227a admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:28.897333 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d7bde6-77e4-4620-a988-c78f7715227a admin admin] PATH_INFO: `/v3/users/1ebb84bf1a924a409a371f5cb9a9460b` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:28.899223 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d7bde6-77e4-4620-a988-c78f7715227a admin admin] RBAC: Authorizing `identity:delete_user(user_id=1ebb84bf1a924a409a371f5cb9a9460b)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:01:28.901952 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d7bde6-77e4-4620-a988-c78f7715227a admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:01:28.922824 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:01:28.934668 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:01:28.986826 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 873/1783] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:01:28 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 341 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 16:01:28.991710 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a62e23-ebb6-45c9-a01b-e90acd32ddbb None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:01:28.994507 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-38d7bde6-77e4-4620-a988-c78f7715227a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1ebb84bf1a924a409a371f5cb9a9460b'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:01:29.005116 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a62e23-ebb6-45c9-a01b-e90acd32ddbb None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:01:29.005356 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-38d7bde6-77e4-4620-a988-c78f7715227a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1ebb84bf1a924a409a371f5cb9a9460b'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:01:29.006176 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-38d7bde6-77e4-4620-a988-c78f7715227a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1ebb84bf1a924a409a371f5cb9a9460b'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:01:29.006854 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-38d7bde6-77e4-4620-a988-c78f7715227a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1ebb84bf1a924a409a371f5cb9a9460b'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:01:29.011310 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2ea710-cc73-434b-a9c7-0218b14134e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:01:29.011993 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2ea710-cc73-434b-a9c7-0218b14134e8 admin admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:29.012362 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2ea710-cc73-434b-a9c7-0218b14134e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:29.012573 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2ea710-cc73-434b-a9c7-0218b14134e8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:29.013256 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2ea710-cc73-434b-a9c7-0218b14134e8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:01:29.015446 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2ea710-cc73-434b-a9c7-0218b14134e8 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:01:29.023091 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-38d7bde6-77e4-4620-a988-c78f7715227a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1ebb84bf1a924a409a371f5cb9a9460b'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:01:29.041407 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 911/1784] 10.222.0.46 () {58 vars in 1223 bytes} [Thu Jun 23 16:01:28 2022] DELETE /identity/v3/users/1ebb84bf1a924a409a371f5cb9a9460b => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 16:01:29.046163 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4cfda3-497b-458f-b829-b272af3197e2 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:29.046639 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4cfda3-497b-458f-b829-b272af3197e2 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:29.047039 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4cfda3-497b-458f-b829-b272af3197e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:29.052090 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 874/1785] 10.222.0.46 () {60 vars in 1152 bytes} [Thu Jun 23 16:01:28 2022] POST /identity/v3/projects => generated 403 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 16:01:29.058714 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2ea710-cc73-434b-a9c7-0218b14134e8 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:01:29.060751 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-0f2ea710-cc73-434b-a9c7-0218b14134e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:01:29.070368 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-7c4cfda3-497b-458f-b829-b272af3197e2 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:01:29.072752 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-7c4cfda3-497b-458f-b829-b272af3197e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:01:29.186721 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 912/1786] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:01:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 16:01:29.201494 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2ea710-cc73-434b-a9c7-0218b14134e8 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:01:29.207611 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4990ce91-61ad-4c09-8b39-7c60d045f600 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:01:29.208489 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4990ce91-61ad-4c09-8b39-7c60d045f600 admin admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:29.208877 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4990ce91-61ad-4c09-8b39-7c60d045f600 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:29.209657 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4990ce91-61ad-4c09-8b39-7c60d045f600 admin admin] PATH_INFO: `/v3/users` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:29.210135 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4990ce91-61ad-4c09-8b39-7c60d045f600 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:01:29.213065 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4990ce91-61ad-4c09-8b39-7c60d045f600 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:01:29.260102 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4cfda3-497b-458f-b829-b272af3197e2 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:01:29.263873 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-7c4cfda3-497b-458f-b829-b272af3197e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:01:29.287401 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 875/1787] 10.222.0.46 () {60 vars in 1143 bytes} [Thu Jun 23 16:01:29 2022] POST /identity/v3/users => generated 308 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 16:01:29.293366 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4990ce91-61ad-4c09-8b39-7c60d045f600 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:01:29.313296 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4990ce91-61ad-4c09-8b39-7c60d045f600 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:01:29.332228 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147ae2f8-7edb-4b50-b78e-5b0c8c30f789 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:01:29.333407 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147ae2f8-7edb-4b50-b78e-5b0c8c30f789 admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:29.334616 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147ae2f8-7edb-4b50-b78e-5b0c8c30f789 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:29.335184 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147ae2f8-7edb-4b50-b78e-5b0c8c30f789 admin admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:29.336183 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147ae2f8-7edb-4b50-b78e-5b0c8c30f789 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:01:29.339167 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147ae2f8-7edb-4b50-b78e-5b0c8c30f789 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:01:29.355823 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 876/1788] 10.222.0.46 () {58 vars in 1121 bytes} [Thu Jun 23 16:01:29 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 16:01:29.361952 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147ae2f8-7edb-4b50-b78e-5b0c8c30f789 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:01:29.374726 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147ae2f8-7edb-4b50-b78e-5b0c8c30f789 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:01:29.380821 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cd466c-166b-4203-b8c9-eac84f561a3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:01:29.381915 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cd466c-166b-4203-b8c9-eac84f561a3d admin admin] REQUEST_METHOD: `PUT` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:29.382339 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cd466c-166b-4203-b8c9-eac84f561a3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:29.382800 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cd466c-166b-4203-b8c9-eac84f561a3d admin admin] PATH_INFO: `/v3/projects/8633f6f54e67428eb4e26da698a37dc1/users/5bc61c7666e94de4a8c663cd76a50135/roles/812d3c567c52469cb2b35709fbdc857c` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:29.402298 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47cd466c-166b-4203-b8c9-eac84f561a3d admin admin] RBAC: Authorizing `identity:create_grant(project_id=8633f6f54e67428eb4e26da698a37dc1, user_id=5bc61c7666e94de4a8c663cd76a50135, role_id=812d3c567c52469cb2b35709fbdc857c)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:01:29.405458 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47cd466c-166b-4203-b8c9-eac84f561a3d admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:01:29.413664 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4cfda3-497b-458f-b829-b272af3197e2 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:01:29.422044 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5913c345-4091-4812-90e6-1c769cb27151 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51784064951a4619ae09b9167fb0fd0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:01:29.423069 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5913c345-4091-4812-90e6-1c769cb27151 service neutron] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:29.423359 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5913c345-4091-4812-90e6-1c769cb27151 service neutron] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:29.423684 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5913c345-4091-4812-90e6-1c769cb27151 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:29.428339 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 877/1789] 10.222.0.46 () {60 vars in 1482 bytes} [Thu Jun 23 16:01:29 2022] PUT /identity/v3/projects/8633f6f54e67428eb4e26da698a37dc1/users/5bc61c7666e94de4a8c663cd76a50135/roles/812d3c567c52469cb2b35709fbdc857c => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 16:01:29.433108 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cd466c-166b-4203-b8c9-eac84f561a3d admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:01:29.444447 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5913c345-4091-4812-90e6-1c769cb27151 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:01:29.446509 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5913c345-4091-4812-90e6-1c769cb27151 service neutron] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:01:29.464047 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 913/1790] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jun 23 16:01:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 16:01:29.535786 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cd466c-166b-4203-b8c9-eac84f561a3d admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:01:29.543865 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f0ed80-d904-4b32-8f70-8a8f512df2b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:01:29.544859 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f0ed80-d904-4b32-8f70-8a8f512df2b8 admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:29.545288 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f0ed80-d904-4b32-8f70-8a8f512df2b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:29.545710 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f0ed80-d904-4b32-8f70-8a8f512df2b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:29.546637 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f0ed80-d904-4b32-8f70-8a8f512df2b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:01:29.548612 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f0ed80-d904-4b32-8f70-8a8f512df2b8 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:01:29.561749 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:01:29.588337 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 878/1791] 10.222.0.46 () {58 vars in 1121 bytes} [Thu Jun 23 16:01:29 2022] GET /identity/v3/roles => generated 1420 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 16:01:29.589210 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5913c345-4091-4812-90e6-1c769cb27151 service neutron] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:01:29.607330 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5913c345-4091-4812-90e6-1c769cb27151 service neutron] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:01:29.617255 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bf385c-1e4c-49f3-9a1d-f4cf967144fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:01:29.619072 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bf385c-1e4c-49f3-9a1d-f4cf967144fb admin admin] REQUEST_METHOD: `PUT` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:29.619723 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bf385c-1e4c-49f3-9a1d-f4cf967144fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:29.620341 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bf385c-1e4c-49f3-9a1d-f4cf967144fb admin admin] PATH_INFO: `/v3/projects/8633f6f54e67428eb4e26da698a37dc1/users/5bc61c7666e94de4a8c663cd76a50135/roles/812d3c567c52469cb2b35709fbdc857c` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:29.625573 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bf385c-1e4c-49f3-9a1d-f4cf967144fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=8633f6f54e67428eb4e26da698a37dc1, user_id=5bc61c7666e94de4a8c663cd76a50135, role_id=812d3c567c52469cb2b35709fbdc857c)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:01:29.630771 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bf385c-1e4c-49f3-9a1d-f4cf967144fb admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:01:29.652690 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 914/1792] 10.222.0.46 () {60 vars in 1482 bytes} [Thu Jun 23 16:01:29 2022] PUT /identity/v3/projects/8633f6f54e67428eb4e26da698a37dc1/users/5bc61c7666e94de4a8c663cd76a50135/roles/812d3c567c52469cb2b35709fbdc857c => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 16:01:30.470577 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f0ed80-d904-4b32-8f70-8a8f512df2b8 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:01:30.475082 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-49f0ed80-d904-4b32-8f70-8a8f512df2b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:01:30.500198 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f0ed80-d904-4b32-8f70-8a8f512df2b8 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:01:30.588538 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dcfc55-99fd-4894-99e9-0cb25585ab55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:01:30.589511 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dcfc55-99fd-4894-99e9-0cb25585ab55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:30.589805 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dcfc55-99fd-4894-99e9-0cb25585ab55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:30.590081 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dcfc55-99fd-4894-99e9-0cb25585ab55 admin admin] PATH_INFO: `/v3/projects/dc9df897282446b5b6c4c1c346d1da7f` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:30.591677 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30dcfc55-99fd-4894-99e9-0cb25585ab55 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dc9df897282446b5b6c4c1c346d1da7f)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:01:30.594960 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30dcfc55-99fd-4894-99e9-0cb25585ab55 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:01:30.683580 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-30dcfc55-99fd-4894-99e9-0cb25585ab55 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dc9df897282446b5b6c4c1c346d1da7f', 'request_id': 'req-30dcfc55-99fd-4894-99e9-0cb25585ab55'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:01:30.691639 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-30dcfc55-99fd-4894-99e9-0cb25585ab55 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dc9df897282446b5b6c4c1c346d1da7f', 'request_id': 'req-30dcfc55-99fd-4894-99e9-0cb25585ab55'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:01:30.707187 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-30dcfc55-99fd-4894-99e9-0cb25585ab55 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dc9df897282446b5b6c4c1c346d1da7f', 'request_id': 'req-30dcfc55-99fd-4894-99e9-0cb25585ab55'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:01:30.725372 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-30dcfc55-99fd-4894-99e9-0cb25585ab55 admin admin] The token cache is being invalidate because project dc9df897282446b5b6c4c1c346d1da7f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80154) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 16:01:30.725765 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-30dcfc55-99fd-4894-99e9-0cb25585ab55 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:01:30.727475 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-30dcfc55-99fd-4894-99e9-0cb25585ab55 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:01:30.729797 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 879/1793] 10.222.0.46 () {58 vars in 1232 bytes} [Thu Jun 23 16:01:30 2022] DELETE /identity/v3/projects/dc9df897282446b5b6c4c1c346d1da7f => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 16:01:30.734184 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bf385c-1e4c-49f3-9a1d-f4cf967144fb admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:01:30.737786 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-d0bf385c-1e4c-49f3-9a1d-f4cf967144fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:01:30.752765 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bf385c-1e4c-49f3-9a1d-f4cf967144fb admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:01:30.785691 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:01:30.858353 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f379d4a9-e6ee-402c-a704-c21297ab7034 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:01:30.859885 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f379d4a9-e6ee-402c-a704-c21297ab7034 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:30.860397 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f379d4a9-e6ee-402c-a704-c21297ab7034 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:30.860691 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f379d4a9-e6ee-402c-a704-c21297ab7034 admin admin] PATH_INFO: `/v3/users/b30b75450fa341fdb8b507e9baabfb4e` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:30.862337 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f379d4a9-e6ee-402c-a704-c21297ab7034 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b30b75450fa341fdb8b507e9baabfb4e)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:01:30.865180 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f379d4a9-e6ee-402c-a704-c21297ab7034 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:01:30.959589 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-f379d4a9-e6ee-402c-a704-c21297ab7034 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b30b75450fa341fdb8b507e9baabfb4e'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:01:30.970887 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-f379d4a9-e6ee-402c-a704-c21297ab7034 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b30b75450fa341fdb8b507e9baabfb4e'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:01:30.971699 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-f379d4a9-e6ee-402c-a704-c21297ab7034 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b30b75450fa341fdb8b507e9baabfb4e'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:01:30.974387 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-f379d4a9-e6ee-402c-a704-c21297ab7034 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b30b75450fa341fdb8b507e9baabfb4e'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:01:30.990915 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-f379d4a9-e6ee-402c-a704-c21297ab7034 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b30b75450fa341fdb8b507e9baabfb4e'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:01:31.018597 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 915/1794] 10.222.0.46 () {58 vars in 1223 bytes} [Thu Jun 23 16:01:30 2022] DELETE /identity/v3/users/b30b75450fa341fdb8b507e9baabfb4e => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 16:01:32.024958 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77ed25e-b705-4bdb-b06f-20de798fedad None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:32.026057 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77ed25e-b705-4bdb-b06f-20de798fedad None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:32.026457 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77ed25e-b705-4bdb-b06f-20de798fedad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:32.079367 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-b77ed25e-b705-4bdb-b06f-20de798fedad None None] MFA Rules not processed for user `5bc61c7666e94de4a8c663cd76a50135`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:01:32.081720 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-b77ed25e-b705-4bdb-b06f-20de798fedad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:01:32.089249 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f379d4a9-e6ee-402c-a704-c21297ab7034 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:01:32.091481 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-f379d4a9-e6ee-402c-a704-c21297ab7034 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:01:32.237833 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f379d4a9-e6ee-402c-a704-c21297ab7034 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:01:32.247830 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e238035b-62ff-4bdf-b7e0-9db6c3997d7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:01:32.249419 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e238035b-62ff-4bdf-b7e0-9db6c3997d7e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:32.251637 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e238035b-62ff-4bdf-b7e0-9db6c3997d7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:32.254482 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e238035b-62ff-4bdf-b7e0-9db6c3997d7e admin admin] PATH_INFO: `/v3/projects/06c66571995f4f6cb86a7ff0d0b5a08e` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:32.256691 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e238035b-62ff-4bdf-b7e0-9db6c3997d7e admin admin] RBAC: Authorizing `identity:delete_project(project_id=06c66571995f4f6cb86a7ff0d0b5a08e)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:01:32.258598 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e238035b-62ff-4bdf-b7e0-9db6c3997d7e admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:01:32.288727 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 880/1795] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:01:32 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 16:01:32.345355 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:01:32.355518 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-e238035b-62ff-4bdf-b7e0-9db6c3997d7e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '06c66571995f4f6cb86a7ff0d0b5a08e', 'request_id': 'req-e238035b-62ff-4bdf-b7e0-9db6c3997d7e'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:01:32.371402 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-e238035b-62ff-4bdf-b7e0-9db6c3997d7e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '06c66571995f4f6cb86a7ff0d0b5a08e', 'request_id': 'req-e238035b-62ff-4bdf-b7e0-9db6c3997d7e'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:01:32.380819 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-e238035b-62ff-4bdf-b7e0-9db6c3997d7e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '06c66571995f4f6cb86a7ff0d0b5a08e', 'request_id': 'req-e238035b-62ff-4bdf-b7e0-9db6c3997d7e'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:01:32.399746 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-e238035b-62ff-4bdf-b7e0-9db6c3997d7e admin admin] The token cache is being invalidate because project 06c66571995f4f6cb86a7ff0d0b5a08e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80153) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 16:01:32.400041 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-e238035b-62ff-4bdf-b7e0-9db6c3997d7e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:01:32.401387 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-e238035b-62ff-4bdf-b7e0-9db6c3997d7e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:01:32.403465 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 916/1796] 10.222.0.46 () {58 vars in 1232 bytes} [Thu Jun 23 16:01:32 2022] DELETE /identity/v3/projects/06c66571995f4f6cb86a7ff0d0b5a08e => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 16:01:32.410766 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368231f2-a493-4133-a3fb-c7cad9e409c1 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:32.411312 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368231f2-a493-4133-a3fb-c7cad9e409c1 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:32.411666 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368231f2-a493-4133-a3fb-c7cad9e409c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:32.435977 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-368231f2-a493-4133-a3fb-c7cad9e409c1 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:01:32.438123 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-368231f2-a493-4133-a3fb-c7cad9e409c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:01:32.454677 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:01:32.518683 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 881/1797] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:01:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 16:01:32.526844 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dbc433-3cc4-43fc-850c-05178b6e7309 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:32.527212 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dbc433-3cc4-43fc-850c-05178b6e7309 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:32.527580 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dbc433-3cc4-43fc-850c-05178b6e7309 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:32.554657 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-65dbc433-3cc4-43fc-850c-05178b6e7309 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:01:32.556544 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-65dbc433-3cc4-43fc-850c-05178b6e7309 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:01:32.565887 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 917/1798] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:01:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 16:01:32.572363 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368231f2-a493-4133-a3fb-c7cad9e409c1 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:01:32.586312 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368231f2-a493-4133-a3fb-c7cad9e409c1 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:01:32.596130 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f1e934-bdee-4f5f-a9d0-44a910943b92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:01:32.597289 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f1e934-bdee-4f5f-a9d0-44a910943b92 admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:32.597822 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f1e934-bdee-4f5f-a9d0-44a910943b92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:32.598157 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f1e934-bdee-4f5f-a9d0-44a910943b92 admin admin] PATH_INFO: `/v3/domains` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:32.598941 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f1e934-bdee-4f5f-a9d0-44a910943b92 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 16:01:32.599275 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f1e934-bdee-4f5f-a9d0-44a910943b92 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:01:32.600985 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f1e934-bdee-4f5f-a9d0-44a910943b92 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:01:32.616866 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 882/1799] 10.222.0.46 () {58 vars in 1152 bytes} [Thu Jun 23 16:01:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 16:01:32.621382 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65dbc433-3cc4-43fc-850c-05178b6e7309 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:01:32.637072 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65dbc433-3cc4-43fc-850c-05178b6e7309 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:01:32.642307 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ed5213-a066-4ba1-9839-45ce6067cc7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:01:32.643048 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ed5213-a066-4ba1-9839-45ce6067cc7e admin admin] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:32.643339 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ed5213-a066-4ba1-9839-45ce6067cc7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:32.643618 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ed5213-a066-4ba1-9839-45ce6067cc7e admin admin] PATH_INFO: `/v3/projects` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:32.644352 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ed5213-a066-4ba1-9839-45ce6067cc7e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:01:32.646359 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ed5213-a066-4ba1-9839-45ce6067cc7e admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:01:32.686529 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 918/1800] 10.222.0.46 () {60 vars in 1152 bytes} [Thu Jun 23 16:01:32 2022] POST /identity/v3/projects => generated 387 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 16:01:32.691441 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f1e934-bdee-4f5f-a9d0-44a910943b92 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:01:32.707111 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f1e934-bdee-4f5f-a9d0-44a910943b92 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:01:32.788428 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b1d3cf-c2d7-4192-8bef-2dc9438ae6c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:01:32.789509 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b1d3cf-c2d7-4192-8bef-2dc9438ae6c9 admin admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:32.789995 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b1d3cf-c2d7-4192-8bef-2dc9438ae6c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:32.791070 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b1d3cf-c2d7-4192-8bef-2dc9438ae6c9 admin admin] PATH_INFO: `/v3/users` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:32.792458 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b1d3cf-c2d7-4192-8bef-2dc9438ae6c9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:01:32.796580 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b1d3cf-c2d7-4192-8bef-2dc9438ae6c9 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:01:32.829922 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 883/1801] 10.222.0.46 () {60 vars in 1143 bytes} [Thu Jun 23 16:01:32 2022] POST /identity/v3/users => generated 294 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 16:01:32.835713 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ed5213-a066-4ba1-9839-45ce6067cc7e admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:01:32.849583 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ed5213-a066-4ba1-9839-45ce6067cc7e admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:01:32.856483 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8af95cc-5213-4ce1-a587-96a19a727d16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:01:32.857843 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8af95cc-5213-4ce1-a587-96a19a727d16 admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:32.858535 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8af95cc-5213-4ce1-a587-96a19a727d16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:32.859085 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8af95cc-5213-4ce1-a587-96a19a727d16 admin admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:32.860488 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8af95cc-5213-4ce1-a587-96a19a727d16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:01:32.863459 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8af95cc-5213-4ce1-a587-96a19a727d16 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:01:32.893837 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 919/1802] 10.222.0.46 () {58 vars in 1121 bytes} [Thu Jun 23 16:01:32 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 16:01:32.895975 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b1d3cf-c2d7-4192-8bef-2dc9438ae6c9 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:01:32.913692 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b1d3cf-c2d7-4192-8bef-2dc9438ae6c9 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:01:32.920560 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b475f933-d865-4ef0-98a3-0cbe213a62d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:01:32.921417 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b475f933-d865-4ef0-98a3-0cbe213a62d1 admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:32.921730 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b475f933-d865-4ef0-98a3-0cbe213a62d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:32.922125 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b475f933-d865-4ef0-98a3-0cbe213a62d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:32.922808 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b475f933-d865-4ef0-98a3-0cbe213a62d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:01:32.924531 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b475f933-d865-4ef0-98a3-0cbe213a62d1 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:01:32.943066 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 884/1803] 10.222.0.46 () {58 vars in 1121 bytes} [Thu Jun 23 16:01:32 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 16:01:32.943835 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8af95cc-5213-4ce1-a587-96a19a727d16 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:01:32.969372 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8af95cc-5213-4ce1-a587-96a19a727d16 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:01:32.979211 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc954672-3edc-4581-bbd4-daf7a2d9a0f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:01:32.980811 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc954672-3edc-4581-bbd4-daf7a2d9a0f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:32.981374 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc954672-3edc-4581-bbd4-daf7a2d9a0f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:32.982415 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc954672-3edc-4581-bbd4-daf7a2d9a0f9 admin admin] PATH_INFO: `/v3/projects/44e96464c5664cc5b433c6aa4bc882dd/users/c16600ec627f40b3a6aad3cff52bf3cb/roles/e2a5bbf3200a4d4091d72bc7ff78c34d` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:33.007231 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc954672-3edc-4581-bbd4-daf7a2d9a0f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=44e96464c5664cc5b433c6aa4bc882dd, user_id=c16600ec627f40b3a6aad3cff52bf3cb, role_id=e2a5bbf3200a4d4091d72bc7ff78c34d)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:01:33.013378 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc954672-3edc-4581-bbd4-daf7a2d9a0f9 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:01:33.033588 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 920/1804] 10.222.0.46 () {60 vars in 1482 bytes} [Thu Jun 23 16:01:32 2022] PUT /identity/v3/projects/44e96464c5664cc5b433c6aa4bc882dd/users/c16600ec627f40b3a6aad3cff52bf3cb/roles/e2a5bbf3200a4d4091d72bc7ff78c34d => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 16:01:33.044096 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b48eb13-b159-41ff-a0a0-6aed7ac73f13 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:33.044872 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b48eb13-b159-41ff-a0a0-6aed7ac73f13 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:33.044872 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b48eb13-b159-41ff-a0a0-6aed7ac73f13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:33.080081 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-0b48eb13-b159-41ff-a0a0-6aed7ac73f13 None None] MFA Rules not processed for user `c16600ec627f40b3a6aad3cff52bf3cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:01:33.082460 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-0b48eb13-b159-41ff-a0a0-6aed7ac73f13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:01:33.189489 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 885/1805] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:01:33 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 16:01:33.194816 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc954672-3edc-4581-bbd4-daf7a2d9a0f9 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:01:33.310435 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc954672-3edc-4581-bbd4-daf7a2d9a0f9 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:01:33.323547 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bb327d-cdbd-4df6-9628-a2a749131d03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:01:33.325138 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bb327d-cdbd-4df6-9628-a2a749131d03 admin admin] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:33.326113 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bb327d-cdbd-4df6-9628-a2a749131d03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:33.326546 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bb327d-cdbd-4df6-9628-a2a749131d03 admin admin] PATH_INFO: `/v3/projects` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:33.327739 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83bb327d-cdbd-4df6-9628-a2a749131d03 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:01:33.331075 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83bb327d-cdbd-4df6-9628-a2a749131d03 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:01:33.376448 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764ee07f-1f20-44e0-942a-1c9a9c70fe56 None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:33.377936 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764ee07f-1f20-44e0-942a-1c9a9c70fe56 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:33.378828 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764ee07f-1f20-44e0-942a-1c9a9c70fe56 None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:33.383470 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 886/1806] 10.222.0.46 () {54 vars in 921 bytes} [Thu Jun 23 16:01:33 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 23 16:01:33.387840 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 921/1807] 10.222.0.46 () {60 vars in 1152 bytes} [Thu Jun 23 16:01:33 2022] POST /identity/v3/projects => generated 387 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 16:01:33.392061 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764ee07f-1f20-44e0-942a-1c9a9c70fe56 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:01:33.402662 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d198ddb-e2f3-446b-9907-ff781d5f4c95 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:33.403626 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d198ddb-e2f3-446b-9907-ff781d5f4c95 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:33.405189 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d198ddb-e2f3-446b-9907-ff781d5f4c95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:33.421207 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:01:33.443848 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-9d198ddb-e2f3-446b-9907-ff781d5f4c95 None None] MFA Rules not processed for user `3f66988b73644fe38b9849663ee70f25`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:01:33.447160 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-9d198ddb-e2f3-446b-9907-ff781d5f4c95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:01:33.579866 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764ee07f-1f20-44e0-942a-1c9a9c70fe56 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:01:33.588284 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b174bcc-745a-4e4a-a540-6b898934ac59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:01:33.588996 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b174bcc-745a-4e4a-a540-6b898934ac59 admin admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:33.589422 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b174bcc-745a-4e4a-a540-6b898934ac59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:33.589711 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b174bcc-745a-4e4a-a540-6b898934ac59 admin admin] PATH_INFO: `/v3/users` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:33.590436 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b174bcc-745a-4e4a-a540-6b898934ac59 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:01:33.592804 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b174bcc-745a-4e4a-a540-6b898934ac59 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:01:33.605435 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:01:33.646801 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 922/1808] 10.222.0.46 () {58 vars in 1018 bytes} [Thu Jun 23 16:01:33 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 16:01:33.649288 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 887/1809] 10.222.0.46 () {60 vars in 1143 bytes} [Thu Jun 23 16:01:33 2022] POST /identity/v3/users => generated 300 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 16:01:33.654463 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d198ddb-e2f3-446b-9907-ff781d5f4c95 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:01:33.668577 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b174bcc-745a-4e4a-a540-6b898934ac59 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:01:33.671385 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-8b174bcc-745a-4e4a-a540-6b898934ac59 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:01:33.683759 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d198ddb-e2f3-446b-9907-ff781d5f4c95 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:01:33.692998 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b174bcc-745a-4e4a-a540-6b898934ac59 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:01:33.697534 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908a3449-6316-47bd-8cb8-324bfc15ff09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:01:33.699293 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908a3449-6316-47bd-8cb8-324bfc15ff09 admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:33.700128 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908a3449-6316-47bd-8cb8-324bfc15ff09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:33.700966 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908a3449-6316-47bd-8cb8-324bfc15ff09 admin admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:33.701158 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34f0633-36c6-4d38-9df2-4e178d92aff5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f66988b73644fe38b9849663ee70f25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:01:33.701833 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34f0633-36c6-4d38-9df2-4e178d92aff5 service placement] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:33.702228 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34f0633-36c6-4d38-9df2-4e178d92aff5 service placement] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:33.703344 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34f0633-36c6-4d38-9df2-4e178d92aff5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:33.705577 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908a3449-6316-47bd-8cb8-324bfc15ff09 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:01:33.709804 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908a3449-6316-47bd-8cb8-324bfc15ff09 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:01:33.727984 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34f0633-36c6-4d38-9df2-4e178d92aff5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:01:33.731401 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34f0633-36c6-4d38-9df2-4e178d92aff5 service placement] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:01:33.734022 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 923/1810] 10.222.0.46 () {58 vars in 1121 bytes} [Thu Jun 23 16:01:33 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 16:01:33.737623 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908a3449-6316-47bd-8cb8-324bfc15ff09 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:01:33.751131 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 888/1811] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jun 23 16:01:33 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 16:01:33.757376 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908a3449-6316-47bd-8cb8-324bfc15ff09 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:01:33.763582 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01360fed-9486-4b25-a79c-068b14ebb3f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:01:33.764490 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01360fed-9486-4b25-a79c-068b14ebb3f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:33.764775 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01360fed-9486-4b25-a79c-068b14ebb3f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:33.765177 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01360fed-9486-4b25-a79c-068b14ebb3f8 admin admin] PATH_INFO: `/v3/projects/a19482dbc84249bd8b055aaca2032b43/users/b9bebb4038ac494aac8b8413bcf12540/roles/812d3c567c52469cb2b35709fbdc857c` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:33.786844 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01360fed-9486-4b25-a79c-068b14ebb3f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a19482dbc84249bd8b055aaca2032b43, user_id=b9bebb4038ac494aac8b8413bcf12540, role_id=812d3c567c52469cb2b35709fbdc857c)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:01:33.790534 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01360fed-9486-4b25-a79c-068b14ebb3f8 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:01:33.811474 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 924/1812] 10.222.0.46 () {60 vars in 1482 bytes} [Thu Jun 23 16:01:33 2022] PUT /identity/v3/projects/a19482dbc84249bd8b055aaca2032b43/users/b9bebb4038ac494aac8b8413bcf12540/roles/812d3c567c52469cb2b35709fbdc857c => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 16:01:33.815849 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34f0633-36c6-4d38-9df2-4e178d92aff5 service placement] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:01:33.828591 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34f0633-36c6-4d38-9df2-4e178d92aff5 service placement] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:01:33.943953 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e16b298-5eed-4458-8ce3-011f5dd4be39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:01:33.945169 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e16b298-5eed-4458-8ce3-011f5dd4be39 admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:33.945746 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e16b298-5eed-4458-8ce3-011f5dd4be39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:33.946798 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e16b298-5eed-4458-8ce3-011f5dd4be39 admin admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:33.948278 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e16b298-5eed-4458-8ce3-011f5dd4be39 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:01:33.951502 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e16b298-5eed-4458-8ce3-011f5dd4be39 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:01:33.974006 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 889/1813] 10.222.0.46 () {58 vars in 1121 bytes} [Thu Jun 23 16:01:33 2022] GET /identity/v3/roles => generated 1420 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 16:01:33.975610 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01360fed-9486-4b25-a79c-068b14ebb3f8 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:01:33.989424 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01360fed-9486-4b25-a79c-068b14ebb3f8 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:01:33.997088 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8018588f-def6-43eb-9c87-314c2d7384f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:01:33.998204 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8018588f-def6-43eb-9c87-314c2d7384f6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:33.998593 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8018588f-def6-43eb-9c87-314c2d7384f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:33.999045 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8018588f-def6-43eb-9c87-314c2d7384f6 admin admin] PATH_INFO: `/v3/projects/a19482dbc84249bd8b055aaca2032b43/users/b9bebb4038ac494aac8b8413bcf12540/roles/812d3c567c52469cb2b35709fbdc857c` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:34.001789 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8018588f-def6-43eb-9c87-314c2d7384f6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a19482dbc84249bd8b055aaca2032b43, user_id=b9bebb4038ac494aac8b8413bcf12540, role_id=812d3c567c52469cb2b35709fbdc857c)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:01:34.004865 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8018588f-def6-43eb-9c87-314c2d7384f6 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:01:34.020109 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 925/1814] 10.222.0.46 () {60 vars in 1482 bytes} [Thu Jun 23 16:01:33 2022] PUT /identity/v3/projects/a19482dbc84249bd8b055aaca2032b43/users/b9bebb4038ac494aac8b8413bcf12540/roles/812d3c567c52469cb2b35709fbdc857c => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 16:01:34.033211 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89bca73-74c6-476d-a8bc-348742fc6fff None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:34.033529 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89bca73-74c6-476d-a8bc-348742fc6fff None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:34.033783 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89bca73-74c6-476d-a8bc-348742fc6fff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:34.080766 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-d89bca73-74c6-476d-a8bc-348742fc6fff None None] MFA Rules not processed for user `b9bebb4038ac494aac8b8413bcf12540`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:01:34.082345 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-d89bca73-74c6-476d-a8bc-348742fc6fff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:01:34.179205 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 890/1815] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:01:34 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 16:01:34.200992 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8018588f-def6-43eb-9c87-314c2d7384f6 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:01:34.203402 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-8018588f-def6-43eb-9c87-314c2d7384f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:01:34.281694 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8018588f-def6-43eb-9c87-314c2d7384f6 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:01:34.287474 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bccea3d-5d0a-4d1d-a6d4-95164dc3e61c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce18eaaba3a6477684fff1fd98ec8b51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:01:34.288193 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bccea3d-5d0a-4d1d-a6d4-95164dc3e61c service cinder] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:34.288499 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bccea3d-5d0a-4d1d-a6d4-95164dc3e61c service cinder] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:34.288788 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bccea3d-5d0a-4d1d-a6d4-95164dc3e61c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:34.301380 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bccea3d-5d0a-4d1d-a6d4-95164dc3e61c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:01:34.302919 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bccea3d-5d0a-4d1d-a6d4-95164dc3e61c service cinder] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:01:34.325413 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 926/1816] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jun 23 16:01:34 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 16:01:34.504519 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac46fe67-417b-4b6e-b09d-71f5d12c7104 None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:34.505097 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac46fe67-417b-4b6e-b09d-71f5d12c7104 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:34.505526 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac46fe67-417b-4b6e-b09d-71f5d12c7104 None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:34.507288 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 891/1817] 10.222.0.46 () {54 vars in 921 bytes} [Thu Jun 23 16:01:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 16:01:34.522500 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe92e7d9-7d49-4123-b8fc-188412d6667a None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:34.522882 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe92e7d9-7d49-4123-b8fc-188412d6667a None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:34.523168 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe92e7d9-7d49-4123-b8fc-188412d6667a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:34.570218 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-fe92e7d9-7d49-4123-b8fc-188412d6667a None None] MFA Rules not processed for user `3f66988b73644fe38b9849663ee70f25`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:01:34.573977 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-fe92e7d9-7d49-4123-b8fc-188412d6667a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:01:34.650226 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac46fe67-417b-4b6e-b09d-71f5d12c7104 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:01:34.653674 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-ac46fe67-417b-4b6e-b09d-71f5d12c7104 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:01:34.681671 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:01:34.888481 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac46fe67-417b-4b6e-b09d-71f5d12c7104 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:01:34.891867 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 927/1818] 10.222.0.46 () {58 vars in 1018 bytes} [Thu Jun 23 16:01:34 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 370 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 16:01:34.897230 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b77d17-eacb-46c3-b239-16ea0fd6209e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:01:34.898820 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b77d17-eacb-46c3-b239-16ea0fd6209e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:34.899289 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b77d17-eacb-46c3-b239-16ea0fd6209e admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:34.899778 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b77d17-eacb-46c3-b239-16ea0fd6209e admin admin] PATH_INFO: `/v3/users/5d8a3488ac9e4c06a7b5f8cba66c0e0d` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:34.902639 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b77d17-eacb-46c3-b239-16ea0fd6209e admin admin] RBAC: Authorizing `identity:delete_user(user_id=5d8a3488ac9e4c06a7b5f8cba66c0e0d)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:01:34.906331 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b77d17-eacb-46c3-b239-16ea0fd6209e admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:01:34.909866 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe92e7d9-7d49-4123-b8fc-188412d6667a None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:01:34.935424 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe92e7d9-7d49-4123-b8fc-188412d6667a None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:01:34.943549 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348f1d16-c470-422e-95a2-fa3e16ac56f9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f66988b73644fe38b9849663ee70f25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:01:34.944407 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348f1d16-c470-422e-95a2-fa3e16ac56f9 service placement] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:34.944791 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348f1d16-c470-422e-95a2-fa3e16ac56f9 service placement] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:34.945253 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348f1d16-c470-422e-95a2-fa3e16ac56f9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:34.956376 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348f1d16-c470-422e-95a2-fa3e16ac56f9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:01:34.957942 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348f1d16-c470-422e-95a2-fa3e16ac56f9 service placement] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:01:34.970641 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 928/1819] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jun 23 16:01:34 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 16:01:35.052393 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-b4b77d17-eacb-46c3-b239-16ea0fd6209e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5d8a3488ac9e4c06a7b5f8cba66c0e0d'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:01:35.053751 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-b4b77d17-eacb-46c3-b239-16ea0fd6209e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5d8a3488ac9e4c06a7b5f8cba66c0e0d'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:01:35.069865 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-b4b77d17-eacb-46c3-b239-16ea0fd6209e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5d8a3488ac9e4c06a7b5f8cba66c0e0d'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:01:35.081989 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-b4b77d17-eacb-46c3-b239-16ea0fd6209e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5d8a3488ac9e4c06a7b5f8cba66c0e0d'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:01:35.083806 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-b4b77d17-eacb-46c3-b239-16ea0fd6209e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5d8a3488ac9e4c06a7b5f8cba66c0e0d'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:01:35.112447 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 892/1820] 10.222.0.46 () {58 vars in 1223 bytes} [Thu Jun 23 16:01:34 2022] DELETE /identity/v3/users/5d8a3488ac9e4c06a7b5f8cba66c0e0d => generated 0 bytes in 471 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 16:01:35.325018 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348f1d16-c470-422e-95a2-fa3e16ac56f9 service placement] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:01:35.330092 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b77d17-eacb-46c3-b239-16ea0fd6209e admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:01:35.333160 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-b4b77d17-eacb-46c3-b239-16ea0fd6209e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:01:35.346771 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348f1d16-c470-422e-95a2-fa3e16ac56f9 service placement] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:01:35.415833 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:01:35.459414 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b77d17-eacb-46c3-b239-16ea0fd6209e admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:01:35.467524 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db21783-22e3-4e75-be96-8e0c7bfbfdbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:01:35.469287 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db21783-22e3-4e75-be96-8e0c7bfbfdbe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:35.469793 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db21783-22e3-4e75-be96-8e0c7bfbfdbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:35.470311 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db21783-22e3-4e75-be96-8e0c7bfbfdbe admin admin] PATH_INFO: `/v3/projects/8c0ad2168cdb4120b8a5a8862fda01d3` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:35.472444 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db21783-22e3-4e75-be96-8e0c7bfbfdbe admin admin] RBAC: Authorizing `identity:delete_project(project_id=8c0ad2168cdb4120b8a5a8862fda01d3)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:01:35.472647 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29145755-f35a-43e2-ac04-0b4a5c61f006 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:01:35.473580 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29145755-f35a-43e2-ac04-0b4a5c61f006 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:35.473857 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29145755-f35a-43e2-ac04-0b4a5c61f006 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:35.474138 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29145755-f35a-43e2-ac04-0b4a5c61f006 admin admin] PATH_INFO: `/v3/users/c16600ec627f40b3a6aad3cff52bf3cb` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:35.475648 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29145755-f35a-43e2-ac04-0b4a5c61f006 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c16600ec627f40b3a6aad3cff52bf3cb)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:01:35.475825 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db21783-22e3-4e75-be96-8e0c7bfbfdbe admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:01:35.478523 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29145755-f35a-43e2-ac04-0b4a5c61f006 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:01:35.535210 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:01:35.548251 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-29145755-f35a-43e2-ac04-0b4a5c61f006 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c16600ec627f40b3a6aad3cff52bf3cb'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:01:35.566642 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-29145755-f35a-43e2-ac04-0b4a5c61f006 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c16600ec627f40b3a6aad3cff52bf3cb'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:01:35.567531 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-0db21783-22e3-4e75-be96-8e0c7bfbfdbe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8c0ad2168cdb4120b8a5a8862fda01d3', 'request_id': 'req-0db21783-22e3-4e75-be96-8e0c7bfbfdbe'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:01:35.578105 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-0db21783-22e3-4e75-be96-8e0c7bfbfdbe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8c0ad2168cdb4120b8a5a8862fda01d3', 'request_id': 'req-0db21783-22e3-4e75-be96-8e0c7bfbfdbe'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:01:35.579677 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-29145755-f35a-43e2-ac04-0b4a5c61f006 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c16600ec627f40b3a6aad3cff52bf3cb'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:01:35.583743 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-29145755-f35a-43e2-ac04-0b4a5c61f006 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c16600ec627f40b3a6aad3cff52bf3cb'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:01:35.590150 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-0db21783-22e3-4e75-be96-8e0c7bfbfdbe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8c0ad2168cdb4120b8a5a8862fda01d3', 'request_id': 'req-0db21783-22e3-4e75-be96-8e0c7bfbfdbe'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:01:35.604919 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-0db21783-22e3-4e75-be96-8e0c7bfbfdbe admin admin] The token cache is being invalidate because project 8c0ad2168cdb4120b8a5a8862fda01d3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80154) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 16:01:35.605369 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-0db21783-22e3-4e75-be96-8e0c7bfbfdbe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:01:35.606243 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-0db21783-22e3-4e75-be96-8e0c7bfbfdbe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:01:35.608559 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 893/1821] 10.222.0.46 () {58 vars in 1232 bytes} [Thu Jun 23 16:01:35 2022] DELETE /identity/v3/projects/8c0ad2168cdb4120b8a5a8862fda01d3 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 16:01:35.612409 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db21783-22e3-4e75-be96-8e0c7bfbfdbe admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:01:35.613998 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-0db21783-22e3-4e75-be96-8e0c7bfbfdbe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:01:35.614524 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-29145755-f35a-43e2-ac04-0b4a5c61f006 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c16600ec627f40b3a6aad3cff52bf3cb'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:01:35.638461 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 929/1822] 10.222.0.46 () {58 vars in 1223 bytes} [Thu Jun 23 16:01:35 2022] DELETE /identity/v3/users/c16600ec627f40b3a6aad3cff52bf3cb => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 16:01:35.644265 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe9922d-1a3a-42fb-b0a4-7bade496317c None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:35.644744 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe9922d-1a3a-42fb-b0a4-7bade496317c None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:35.645216 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe9922d-1a3a-42fb-b0a4-7bade496317c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:35.683445 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-1fe9922d-1a3a-42fb-b0a4-7bade496317c None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:01:35.693317 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-1fe9922d-1a3a-42fb-b0a4-7bade496317c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:01:35.728355 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db21783-22e3-4e75-be96-8e0c7bfbfdbe admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:01:35.820838 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 930/1823] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:01:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 16:01:35.830102 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe9922d-1a3a-42fb-b0a4-7bade496317c None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:01:35.838147 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-1fe9922d-1a3a-42fb-b0a4-7bade496317c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:01:35.868827 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8431f94-ea22-4ff1-b58a-a8b2a25e64ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:01:35.869981 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8431f94-ea22-4ff1-b58a-a8b2a25e64ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:35.870269 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8431f94-ea22-4ff1-b58a-a8b2a25e64ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:35.870546 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8431f94-ea22-4ff1-b58a-a8b2a25e64ec admin admin] PATH_INFO: `/v3/users/5bc61c7666e94de4a8c663cd76a50135` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:35.872205 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8431f94-ea22-4ff1-b58a-a8b2a25e64ec admin admin] RBAC: Authorizing `identity:delete_user(user_id=5bc61c7666e94de4a8c663cd76a50135)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:01:35.874292 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8431f94-ea22-4ff1-b58a-a8b2a25e64ec admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:01:35.972420 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-b8431f94-ea22-4ff1-b58a-a8b2a25e64ec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5bc61c7666e94de4a8c663cd76a50135'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:01:35.973632 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-b8431f94-ea22-4ff1-b58a-a8b2a25e64ec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5bc61c7666e94de4a8c663cd76a50135'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:01:35.976652 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:01:35.986253 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-b8431f94-ea22-4ff1-b58a-a8b2a25e64ec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5bc61c7666e94de4a8c663cd76a50135'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:01:36.002489 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe9922d-1a3a-42fb-b0a4-7bade496317c None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:01:36.006333 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-b8431f94-ea22-4ff1-b58a-a8b2a25e64ec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5bc61c7666e94de4a8c663cd76a50135'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:01:36.007170 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-b8431f94-ea22-4ff1-b58a-a8b2a25e64ec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5bc61c7666e94de4a8c663cd76a50135'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:01:36.012125 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5793ec03-1a74-4f2f-9727-e722940f5c5f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51784064951a4619ae09b9167fb0fd0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:01:36.015138 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5793ec03-1a74-4f2f-9727-e722940f5c5f service neutron] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:36.015645 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5793ec03-1a74-4f2f-9727-e722940f5c5f service neutron] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:36.016219 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5793ec03-1a74-4f2f-9727-e722940f5c5f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:36.019524 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-5793ec03-1a74-4f2f-9727-e722940f5c5f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:01:36.022037 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:01:36.046523 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5793ec03-1a74-4f2f-9727-e722940f5c5f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:01:36.049518 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5793ec03-1a74-4f2f-9727-e722940f5c5f service neutron] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:01:36.055130 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 894/1824] 10.222.0.46 () {58 vars in 1223 bytes} [Thu Jun 23 16:01:35 2022] DELETE /identity/v3/users/5bc61c7666e94de4a8c663cd76a50135 => generated 0 bytes in 443 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 16:01:36.070069 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 931/1825] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jun 23 16:01:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 240 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 16:01:36.424201 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8431f94-ea22-4ff1-b58a-a8b2a25e64ec admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:01:36.425409 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-b8431f94-ea22-4ff1-b58a-a8b2a25e64ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:01:36.540774 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8431f94-ea22-4ff1-b58a-a8b2a25e64ec admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:01:36.547594 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9072d15e-57d7-453d-8815-79484e9bb266 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:01:36.548670 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9072d15e-57d7-453d-8815-79484e9bb266 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:36.548944 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9072d15e-57d7-453d-8815-79484e9bb266 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:36.549229 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9072d15e-57d7-453d-8815-79484e9bb266 admin admin] PATH_INFO: `/v3/projects/8633f6f54e67428eb4e26da698a37dc1` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:36.551714 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9072d15e-57d7-453d-8815-79484e9bb266 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8633f6f54e67428eb4e26da698a37dc1)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:01:36.554153 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9072d15e-57d7-453d-8815-79484e9bb266 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:01:36.621149 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5793ec03-1a74-4f2f-9727-e722940f5c5f service neutron] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:01:36.624832 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-5793ec03-1a74-4f2f-9727-e722940f5c5f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:01:36.638382 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-9072d15e-57d7-453d-8815-79484e9bb266 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8633f6f54e67428eb4e26da698a37dc1', 'request_id': 'req-9072d15e-57d7-453d-8815-79484e9bb266'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:01:36.644746 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-9072d15e-57d7-453d-8815-79484e9bb266 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8633f6f54e67428eb4e26da698a37dc1', 'request_id': 'req-9072d15e-57d7-453d-8815-79484e9bb266'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:01:36.655098 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-9072d15e-57d7-453d-8815-79484e9bb266 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8633f6f54e67428eb4e26da698a37dc1', 'request_id': 'req-9072d15e-57d7-453d-8815-79484e9bb266'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:01:36.664729 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5793ec03-1a74-4f2f-9727-e722940f5c5f service neutron] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:01:36.670547 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-9072d15e-57d7-453d-8815-79484e9bb266 admin admin] The token cache is being invalidate because project 8633f6f54e67428eb4e26da698a37dc1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80154) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 16:01:36.670877 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-9072d15e-57d7-453d-8815-79484e9bb266 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:01:36.671560 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-9072d15e-57d7-453d-8815-79484e9bb266 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:01:36.673272 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 895/1826] 10.222.0.46 () {58 vars in 1232 bytes} [Thu Jun 23 16:01:36 2022] DELETE /identity/v3/projects/8633f6f54e67428eb4e26da698a37dc1 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 16:01:36.678938 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03aa1dbe-e1a5-4a96-bf11-67d5f4773ec4 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:36.679402 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03aa1dbe-e1a5-4a96-bf11-67d5f4773ec4 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:36.679980 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03aa1dbe-e1a5-4a96-bf11-67d5f4773ec4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:36.720984 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-03aa1dbe-e1a5-4a96-bf11-67d5f4773ec4 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:01:36.723935 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-03aa1dbe-e1a5-4a96-bf11-67d5f4773ec4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:01:36.800179 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407d3112-7e53-4681-a366-053988afe259 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:01:36.801878 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407d3112-7e53-4681-a366-053988afe259 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:36.802354 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407d3112-7e53-4681-a366-053988afe259 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:36.802919 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407d3112-7e53-4681-a366-053988afe259 admin admin] PATH_INFO: `/v3/projects/44e96464c5664cc5b433c6aa4bc882dd` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:36.805190 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407d3112-7e53-4681-a366-053988afe259 admin admin] RBAC: Authorizing `identity:delete_project(project_id=44e96464c5664cc5b433c6aa4bc882dd)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:01:36.809192 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407d3112-7e53-4681-a366-053988afe259 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:01:36.855267 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 896/1827] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:01:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 16:01:36.866484 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b28656-5a48-47d8-9cbe-3f0e035ab28b None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:36.867003 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b28656-5a48-47d8-9cbe-3f0e035ab28b None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:36.867298 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b28656-5a48-47d8-9cbe-3f0e035ab28b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:36.895395 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-f5b28656-5a48-47d8-9cbe-3f0e035ab28b None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:01:36.898300 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-f5b28656-5a48-47d8-9cbe-3f0e035ab28b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:01:36.912301 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:01:36.917913 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-407d3112-7e53-4681-a366-053988afe259 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '44e96464c5664cc5b433c6aa4bc882dd', 'request_id': 'req-407d3112-7e53-4681-a366-053988afe259'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:01:36.927794 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-407d3112-7e53-4681-a366-053988afe259 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '44e96464c5664cc5b433c6aa4bc882dd', 'request_id': 'req-407d3112-7e53-4681-a366-053988afe259'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:01:36.937379 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-407d3112-7e53-4681-a366-053988afe259 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '44e96464c5664cc5b433c6aa4bc882dd', 'request_id': 'req-407d3112-7e53-4681-a366-053988afe259'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:01:36.955584 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-407d3112-7e53-4681-a366-053988afe259 admin admin] The token cache is being invalidate because project 44e96464c5664cc5b433c6aa4bc882dd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80153) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 16:01:36.956153 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-407d3112-7e53-4681-a366-053988afe259 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:01:36.956886 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-407d3112-7e53-4681-a366-053988afe259 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:01:36.958792 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 932/1828] 10.222.0.46 () {58 vars in 1232 bytes} [Thu Jun 23 16:01:36 2022] DELETE /identity/v3/projects/44e96464c5664cc5b433c6aa4bc882dd => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 16:01:36.965083 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407d3112-7e53-4681-a366-053988afe259 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:01:36.966775 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-407d3112-7e53-4681-a366-053988afe259 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:01:36.973258 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 897/1829] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:01:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 16:01:36.977752 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b28656-5a48-47d8-9cbe-3f0e035ab28b None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:01:36.991424 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b28656-5a48-47d8-9cbe-3f0e035ab28b None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:01:36.993774 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407d3112-7e53-4681-a366-053988afe259 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:01:36.998484 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fc0e2d-62ea-4e4c-b1a8-254308c87ae5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:01:36.999332 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fc0e2d-62ea-4e4c-b1a8-254308c87ae5 admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:37.000127 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fc0e2d-62ea-4e4c-b1a8-254308c87ae5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:37.000450 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fc0e2d-62ea-4e4c-b1a8-254308c87ae5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:37.000803 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eefa43b-9396-4a10-96c8-1d90150b7e4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:01:37.001352 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fc0e2d-62ea-4e4c-b1a8-254308c87ae5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 16:01:37.001674 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fc0e2d-62ea-4e4c-b1a8-254308c87ae5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:01:37.002030 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eefa43b-9396-4a10-96c8-1d90150b7e4d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:37.002275 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eefa43b-9396-4a10-96c8-1d90150b7e4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:37.002564 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eefa43b-9396-4a10-96c8-1d90150b7e4d admin admin] PATH_INFO: `/v3/users/b9bebb4038ac494aac8b8413bcf12540` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:37.003981 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fc0e2d-62ea-4e4c-b1a8-254308c87ae5 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:01:37.004847 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eefa43b-9396-4a10-96c8-1d90150b7e4d admin admin] RBAC: Authorizing `identity:delete_user(user_id=b9bebb4038ac494aac8b8413bcf12540)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:01:37.007584 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eefa43b-9396-4a10-96c8-1d90150b7e4d admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:01:37.021274 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 898/1830] 10.222.0.46 () {58 vars in 1152 bytes} [Thu Jun 23 16:01:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 16:01:37.028219 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fc0e2d-62ea-4e4c-b1a8-254308c87ae5 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:01:37.045710 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fc0e2d-62ea-4e4c-b1a8-254308c87ae5 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:01:37.052060 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755375fb-7eae-4330-8567-0fa38ea6c9d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:01:37.052755 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755375fb-7eae-4330-8567-0fa38ea6c9d3 admin admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:37.053043 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755375fb-7eae-4330-8567-0fa38ea6c9d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:37.053321 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755375fb-7eae-4330-8567-0fa38ea6c9d3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:37.054036 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755375fb-7eae-4330-8567-0fa38ea6c9d3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:01:37.057554 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755375fb-7eae-4330-8567-0fa38ea6c9d3 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:01:37.096609 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 899/1831] 10.222.0.46 () {60 vars in 1152 bytes} [Thu Jun 23 16:01:37 2022] POST /identity/v3/projects => generated 363 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 16:01:37.114974 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755375fb-7eae-4330-8567-0fa38ea6c9d3 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:01:37.123911 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-1eefa43b-9396-4a10-96c8-1d90150b7e4d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b9bebb4038ac494aac8b8413bcf12540'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:01:37.141739 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-1eefa43b-9396-4a10-96c8-1d90150b7e4d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b9bebb4038ac494aac8b8413bcf12540'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:01:37.143137 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-1eefa43b-9396-4a10-96c8-1d90150b7e4d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b9bebb4038ac494aac8b8413bcf12540'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:01:37.144660 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-1eefa43b-9396-4a10-96c8-1d90150b7e4d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b9bebb4038ac494aac8b8413bcf12540'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:01:37.179751 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:01:37.198749 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-1eefa43b-9396-4a10-96c8-1d90150b7e4d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b9bebb4038ac494aac8b8413bcf12540'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:01:37.217705 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755375fb-7eae-4330-8567-0fa38ea6c9d3 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:01:37.223932 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01aa88c-9050-40a3-a4e7-5bd1f7a7a01e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:01:37.225387 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01aa88c-9050-40a3-a4e7-5bd1f7a7a01e admin admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:37.225387 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01aa88c-9050-40a3-a4e7-5bd1f7a7a01e admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:37.225671 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01aa88c-9050-40a3-a4e7-5bd1f7a7a01e admin admin] PATH_INFO: `/v3/users` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:37.226637 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01aa88c-9050-40a3-a4e7-5bd1f7a7a01e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:01:37.228849 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01aa88c-9050-40a3-a4e7-5bd1f7a7a01e admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:01:37.239166 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 933/1832] 10.222.0.46 () {58 vars in 1223 bytes} [Thu Jun 23 16:01:36 2022] DELETE /identity/v3/users/b9bebb4038ac494aac8b8413bcf12540 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 16:01:37.282091 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 900/1833] 10.222.0.46 () {60 vars in 1143 bytes} [Thu Jun 23 16:01:37 2022] POST /identity/v3/users => generated 282 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 16:01:37.288872 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eefa43b-9396-4a10-96c8-1d90150b7e4d admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:01:37.290447 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-1eefa43b-9396-4a10-96c8-1d90150b7e4d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:01:37.425526 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eefa43b-9396-4a10-96c8-1d90150b7e4d admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:01:37.440672 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388ee14d-dcd6-4b64-8afe-ebdbfd38a870 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:01:37.442340 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388ee14d-dcd6-4b64-8afe-ebdbfd38a870 admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:37.442881 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388ee14d-dcd6-4b64-8afe-ebdbfd38a870 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:37.443532 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388ee14d-dcd6-4b64-8afe-ebdbfd38a870 admin admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:37.445052 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388ee14d-dcd6-4b64-8afe-ebdbfd38a870 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:01:37.451545 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388ee14d-dcd6-4b64-8afe-ebdbfd38a870 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:01:37.481114 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 934/1834] 10.222.0.46 () {58 vars in 1121 bytes} [Thu Jun 23 16:01:37 2022] GET /identity/v3/roles => generated 1420 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 16:01:37.489012 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01aa88c-9050-40a3-a4e7-5bd1f7a7a01e admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:01:37.510081 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01aa88c-9050-40a3-a4e7-5bd1f7a7a01e admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:01:37.515230 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acab7f4-37da-48ee-9182-da7df55e0e32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:01:37.515943 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acab7f4-37da-48ee-9182-da7df55e0e32 admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:37.516224 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acab7f4-37da-48ee-9182-da7df55e0e32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:37.516496 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acab7f4-37da-48ee-9182-da7df55e0e32 admin admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:37.517254 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3acab7f4-37da-48ee-9182-da7df55e0e32 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:01:37.519399 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3acab7f4-37da-48ee-9182-da7df55e0e32 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:01:37.531000 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 901/1835] 10.222.0.46 () {58 vars in 1121 bytes} [Thu Jun 23 16:01:37 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 16:01:37.536889 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388ee14d-dcd6-4b64-8afe-ebdbfd38a870 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:01:37.550347 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388ee14d-dcd6-4b64-8afe-ebdbfd38a870 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:01:37.556445 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b261de-d975-4673-a2ba-191e6083795a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:01:37.557385 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b261de-d975-4673-a2ba-191e6083795a admin admin] REQUEST_METHOD: `PUT` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:37.557670 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b261de-d975-4673-a2ba-191e6083795a admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:37.557948 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b261de-d975-4673-a2ba-191e6083795a admin admin] PATH_INFO: `/v3/projects/26dfef1961b54ec798a413ef55c72333/users/c7019da576d64eb595cfddb70e5d2eb3/roles/e2a5bbf3200a4d4091d72bc7ff78c34d` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:37.584748 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b261de-d975-4673-a2ba-191e6083795a admin admin] RBAC: Authorizing `identity:create_grant(project_id=26dfef1961b54ec798a413ef55c72333, user_id=c7019da576d64eb595cfddb70e5d2eb3, role_id=e2a5bbf3200a4d4091d72bc7ff78c34d)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:01:37.591369 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b261de-d975-4673-a2ba-191e6083795a admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:01:37.612149 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 935/1836] 10.222.0.46 () {60 vars in 1482 bytes} [Thu Jun 23 16:01:37 2022] PUT /identity/v3/projects/26dfef1961b54ec798a413ef55c72333/users/c7019da576d64eb595cfddb70e5d2eb3/roles/e2a5bbf3200a4d4091d72bc7ff78c34d => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 16:01:37.626232 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c493fa37-d343-4927-9fdc-fbca9eea75db None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:37.626817 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c493fa37-d343-4927-9fdc-fbca9eea75db None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:37.627319 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c493fa37-d343-4927-9fdc-fbca9eea75db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:37.677497 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-c493fa37-d343-4927-9fdc-fbca9eea75db None None] MFA Rules not processed for user `c7019da576d64eb595cfddb70e5d2eb3`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:01:37.679799 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-c493fa37-d343-4927-9fdc-fbca9eea75db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:01:37.777659 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:01:37.820804 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b261de-d975-4673-a2ba-191e6083795a admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:01:37.822386 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 902/1837] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:01:37 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 16:01:37.839568 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c493fa37-d343-4927-9fdc-fbca9eea75db None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:01:37.842121 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-c493fa37-d343-4927-9fdc-fbca9eea75db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:01:37.913116 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b261de-d975-4673-a2ba-191e6083795a admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:01:37.921113 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3580da9-dfb8-42d3-9295-8d9f77354b72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:01:37.922206 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3580da9-dfb8-42d3-9295-8d9f77354b72 admin admin] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:37.922766 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3580da9-dfb8-42d3-9295-8d9f77354b72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:37.923278 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3580da9-dfb8-42d3-9295-8d9f77354b72 admin admin] PATH_INFO: `/v3/projects` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:37.924412 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3580da9-dfb8-42d3-9295-8d9f77354b72 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:01:37.927840 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3580da9-dfb8-42d3-9295-8d9f77354b72 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:01:37.951759 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c493fa37-d343-4927-9fdc-fbca9eea75db None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:01:37.959331 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785c5cc3-09df-4831-ac37-1bdea33694b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:01:37.960867 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785c5cc3-09df-4831-ac37-1bdea33694b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:37.961372 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785c5cc3-09df-4831-ac37-1bdea33694b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:37.962418 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785c5cc3-09df-4831-ac37-1bdea33694b2 admin admin] PATH_INFO: `/v3/projects/a19482dbc84249bd8b055aaca2032b43` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:37.964893 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785c5cc3-09df-4831-ac37-1bdea33694b2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a19482dbc84249bd8b055aaca2032b43)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:01:37.968712 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785c5cc3-09df-4831-ac37-1bdea33694b2 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:01:37.972737 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 936/1838] 10.222.0.46 () {60 vars in 1152 bytes} [Thu Jun 23 16:01:37 2022] POST /identity/v3/projects => generated 365 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 16:01:37.973838 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3580da9-dfb8-42d3-9295-8d9f77354b72 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:01:38.031807 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-785c5cc3-09df-4831-ac37-1bdea33694b2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a19482dbc84249bd8b055aaca2032b43', 'request_id': 'req-785c5cc3-09df-4831-ac37-1bdea33694b2'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:01:38.041995 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-785c5cc3-09df-4831-ac37-1bdea33694b2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a19482dbc84249bd8b055aaca2032b43', 'request_id': 'req-785c5cc3-09df-4831-ac37-1bdea33694b2'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:01:38.044297 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:01:38.054512 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-785c5cc3-09df-4831-ac37-1bdea33694b2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a19482dbc84249bd8b055aaca2032b43', 'request_id': 'req-785c5cc3-09df-4831-ac37-1bdea33694b2'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:01:38.069571 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3580da9-dfb8-42d3-9295-8d9f77354b72 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:01:38.073683 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-785c5cc3-09df-4831-ac37-1bdea33694b2 admin admin] The token cache is being invalidate because project a19482dbc84249bd8b055aaca2032b43 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80154) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 16:01:38.074035 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-785c5cc3-09df-4831-ac37-1bdea33694b2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:01:38.074891 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-785c5cc3-09df-4831-ac37-1bdea33694b2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:01:38.077289 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 903/1839] 10.222.0.46 () {58 vars in 1232 bytes} [Thu Jun 23 16:01:37 2022] DELETE /identity/v3/projects/a19482dbc84249bd8b055aaca2032b43 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 16:01:38.082635 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6632f4-1efb-4d8f-a461-1b61359d1a28 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:38.082958 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6632f4-1efb-4d8f-a461-1b61359d1a28 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:38.083235 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6632f4-1efb-4d8f-a461-1b61359d1a28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:38.103828 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-5b6632f4-1efb-4d8f-a461-1b61359d1a28 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:01:38.105070 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-5b6632f4-1efb-4d8f-a461-1b61359d1a28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:01:38.159033 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f33aa5-a40b-44ad-a0ea-82254b0db8ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:01:38.159729 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f33aa5-a40b-44ad-a0ea-82254b0db8ab admin admin] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:38.159998 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f33aa5-a40b-44ad-a0ea-82254b0db8ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:38.160280 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f33aa5-a40b-44ad-a0ea-82254b0db8ab admin admin] PATH_INFO: `/v3/users` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:38.160989 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f33aa5-a40b-44ad-a0ea-82254b0db8ab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:01:38.163012 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f33aa5-a40b-44ad-a0ea-82254b0db8ab admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:01:38.185846 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 937/1840] 10.222.0.46 () {60 vars in 1143 bytes} [Thu Jun 23 16:01:37 2022] POST /identity/v3/users => generated 289 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 16:01:38.190995 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 904/1841] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:01:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 16:01:38.191283 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f33aa5-a40b-44ad-a0ea-82254b0db8ab admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:01:38.192360 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-c0f33aa5-a40b-44ad-a0ea-82254b0db8ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:01:38.194362 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdfefcc-6278-4f1e-a6fc-ce5ba9b9721f None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:38.194659 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdfefcc-6278-4f1e-a6fc-ce5ba9b9721f None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:38.194986 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdfefcc-6278-4f1e-a6fc-ce5ba9b9721f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:38.205611 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f33aa5-a40b-44ad-a0ea-82254b0db8ab admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:01:38.210805 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558b5f27-83d6-446d-b44d-f9e6e4233446 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:01:38.212639 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558b5f27-83d6-446d-b44d-f9e6e4233446 admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:38.212639 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558b5f27-83d6-446d-b44d-f9e6e4233446 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:38.212639 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558b5f27-83d6-446d-b44d-f9e6e4233446 admin admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:38.213401 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558b5f27-83d6-446d-b44d-f9e6e4233446 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:01:38.215107 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558b5f27-83d6-446d-b44d-f9e6e4233446 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:01:38.220819 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-fbdfefcc-6278-4f1e-a6fc-ce5ba9b9721f None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:01:38.223154 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-fbdfefcc-6278-4f1e-a6fc-ce5ba9b9721f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:01:38.226830 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 938/1842] 10.222.0.46 () {58 vars in 1121 bytes} [Thu Jun 23 16:01:38 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 16:01:38.231411 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558b5f27-83d6-446d-b44d-f9e6e4233446 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:01:38.235661 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 905/1843] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:01:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 16:01:38.240381 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdfefcc-6278-4f1e-a6fc-ce5ba9b9721f None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:01:38.243516 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558b5f27-83d6-446d-b44d-f9e6e4233446 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:01:38.248801 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac897474-a833-467d-b69c-b56af62b6580 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:01:38.249836 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac897474-a833-467d-b69c-b56af62b6580 admin admin] REQUEST_METHOD: `PUT` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:38.250222 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac897474-a833-467d-b69c-b56af62b6580 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:38.250517 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac897474-a833-467d-b69c-b56af62b6580 admin admin] PATH_INFO: `/v3/projects/05d4d1c45d234e3bbc735192eec32bed/users/91d4ac439fd44121b44498d66df8d00e/roles/812d3c567c52469cb2b35709fbdc857c` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:38.252231 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdfefcc-6278-4f1e-a6fc-ce5ba9b9721f None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:01:38.258533 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac05222b-d77a-466f-8c38-707948c1ae50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:01:38.259787 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac05222b-d77a-466f-8c38-707948c1ae50 admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:38.260065 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac05222b-d77a-466f-8c38-707948c1ae50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:38.260336 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac05222b-d77a-466f-8c38-707948c1ae50 admin admin] PATH_INFO: `/v3/domains` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:38.261084 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac05222b-d77a-466f-8c38-707948c1ae50 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 16:01:38.261392 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac05222b-d77a-466f-8c38-707948c1ae50 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:01:38.263065 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac05222b-d77a-466f-8c38-707948c1ae50 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:01:38.269257 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac897474-a833-467d-b69c-b56af62b6580 admin admin] RBAC: Authorizing `identity:create_grant(project_id=05d4d1c45d234e3bbc735192eec32bed, user_id=91d4ac439fd44121b44498d66df8d00e, role_id=812d3c567c52469cb2b35709fbdc857c)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:01:38.272199 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac897474-a833-467d-b69c-b56af62b6580 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:01:38.273013 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:01:38.278333 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 906/1844] 10.222.0.46 () {58 vars in 1152 bytes} [Thu Jun 23 16:01:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 16:01:38.284760 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac05222b-d77a-466f-8c38-707948c1ae50 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:01:38.290116 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 939/1845] 10.222.0.46 () {60 vars in 1482 bytes} [Thu Jun 23 16:01:38 2022] PUT /identity/v3/projects/05d4d1c45d234e3bbc735192eec32bed/users/91d4ac439fd44121b44498d66df8d00e/roles/812d3c567c52469cb2b35709fbdc857c => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 16:01:38.295053 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac897474-a833-467d-b69c-b56af62b6580 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:01:38.297043 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac05222b-d77a-466f-8c38-707948c1ae50 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:01:38.376364 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb91b8f-e7a6-48c6-b8f2-fcb0ebf05d3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:01:38.377232 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb91b8f-e7a6-48c6-b8f2-fcb0ebf05d3c admin admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:38.377715 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb91b8f-e7a6-48c6-b8f2-fcb0ebf05d3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:38.377997 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb91b8f-e7a6-48c6-b8f2-fcb0ebf05d3c admin admin] PATH_INFO: `/v3/projects` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:38.378821 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb91b8f-e7a6-48c6-b8f2-fcb0ebf05d3c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:01:38.380978 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb91b8f-e7a6-48c6-b8f2-fcb0ebf05d3c admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:01:38.381323 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac897474-a833-467d-b69c-b56af62b6580 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:01:38.391571 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d349c078-0e53-456b-8bd1-fc6e48dc5bfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:01:38.392961 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d349c078-0e53-456b-8bd1-fc6e48dc5bfc admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:38.393640 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d349c078-0e53-456b-8bd1-fc6e48dc5bfc admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:38.394143 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d349c078-0e53-456b-8bd1-fc6e48dc5bfc admin admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:38.396209 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d349c078-0e53-456b-8bd1-fc6e48dc5bfc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:01:38.400704 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d349c078-0e53-456b-8bd1-fc6e48dc5bfc admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:01:38.418783 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 940/1846] 10.222.0.46 () {58 vars in 1121 bytes} [Thu Jun 23 16:01:38 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 16:01:38.420295 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d349c078-0e53-456b-8bd1-fc6e48dc5bfc admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:01:38.426948 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 907/1847] 10.222.0.46 () {60 vars in 1152 bytes} [Thu Jun 23 16:01:38 2022] POST /identity/v3/projects => generated 381 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 16:01:38.428697 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb91b8f-e7a6-48c6-b8f2-fcb0ebf05d3c admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:01:38.434397 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d349c078-0e53-456b-8bd1-fc6e48dc5bfc admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:01:38.537728 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb91b8f-e7a6-48c6-b8f2-fcb0ebf05d3c admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:01:38.538539 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:01:38.547663 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5c64de-19ee-4db9-8032-ce1314d2d41f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:01:38.548582 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5c64de-19ee-4db9-8032-ce1314d2d41f admin admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:38.548926 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5c64de-19ee-4db9-8032-ce1314d2d41f admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:38.549247 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5c64de-19ee-4db9-8032-ce1314d2d41f admin admin] PATH_INFO: `/v3/users` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:38.550032 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5c64de-19ee-4db9-8032-ce1314d2d41f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:01:38.567073 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5c64de-19ee-4db9-8032-ce1314d2d41f admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:01:38.574377 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4fab31-70e1-44f5-adbb-8442e49ec22b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:01:38.575873 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4fab31-70e1-44f5-adbb-8442e49ec22b admin admin] REQUEST_METHOD: `PUT` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:38.576163 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4fab31-70e1-44f5-adbb-8442e49ec22b admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:38.576439 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4fab31-70e1-44f5-adbb-8442e49ec22b admin admin] PATH_INFO: `/v3/projects/05d4d1c45d234e3bbc735192eec32bed/users/91d4ac439fd44121b44498d66df8d00e/roles/812d3c567c52469cb2b35709fbdc857c` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:38.578623 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4fab31-70e1-44f5-adbb-8442e49ec22b admin admin] RBAC: Authorizing `identity:create_grant(project_id=05d4d1c45d234e3bbc735192eec32bed, user_id=91d4ac439fd44121b44498d66df8d00e, role_id=812d3c567c52469cb2b35709fbdc857c)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:01:38.581364 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4fab31-70e1-44f5-adbb-8442e49ec22b admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:01:38.617462 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 941/1848] 10.222.0.46 () {60 vars in 1482 bytes} [Thu Jun 23 16:01:38 2022] PUT /identity/v3/projects/05d4d1c45d234e3bbc735192eec32bed/users/91d4ac439fd44121b44498d66df8d00e/roles/812d3c567c52469cb2b35709fbdc857c => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 16:01:38.626631 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e384adb0-eabf-4787-8c10-52d42cb2449c None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:38.627224 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e384adb0-eabf-4787-8c10-52d42cb2449c None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:38.627661 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e384adb0-eabf-4787-8c10-52d42cb2449c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:38.648772 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 908/1849] 10.222.0.46 () {60 vars in 1143 bytes} [Thu Jun 23 16:01:38 2022] POST /identity/v3/users => generated 291 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 16:01:38.655270 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5c64de-19ee-4db9-8032-ce1314d2d41f admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:01:38.681151 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-e384adb0-eabf-4787-8c10-52d42cb2449c None None] MFA Rules not processed for user `91d4ac439fd44121b44498d66df8d00e`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:01:38.681151 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-e384adb0-eabf-4787-8c10-52d42cb2449c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:01:38.682927 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5c64de-19ee-4db9-8032-ce1314d2d41f admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:01:38.830755 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 942/1850] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:01:38 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 16:01:38.834315 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08445917-94cf-4cd4-b917-d285cdf66855 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:01:38.835543 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08445917-94cf-4cd4-b917-d285cdf66855 admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:38.836032 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08445917-94cf-4cd4-b917-d285cdf66855 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:38.837262 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08445917-94cf-4cd4-b917-d285cdf66855 admin admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:38.837984 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08445917-94cf-4cd4-b917-d285cdf66855 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:01:38.841588 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08445917-94cf-4cd4-b917-d285cdf66855 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:01:38.849598 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e384adb0-eabf-4787-8c10-52d42cb2449c None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:01:38.859631 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-e384adb0-eabf-4787-8c10-52d42cb2449c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:01:38.866933 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 909/1851] 10.222.0.46 () {58 vars in 1121 bytes} [Thu Jun 23 16:01:38 2022] GET /identity/v3/roles => generated 1420 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 16:01:38.872355 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08445917-94cf-4cd4-b917-d285cdf66855 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:01:38.888012 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08445917-94cf-4cd4-b917-d285cdf66855 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:01:38.893963 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61536eb-3bc5-41b9-a771-d13b98d9509c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:01:38.894991 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61536eb-3bc5-41b9-a771-d13b98d9509c admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:38.895290 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61536eb-3bc5-41b9-a771-d13b98d9509c admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:38.895554 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61536eb-3bc5-41b9-a771-d13b98d9509c admin admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:38.896692 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61536eb-3bc5-41b9-a771-d13b98d9509c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:01:38.898829 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61536eb-3bc5-41b9-a771-d13b98d9509c admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:01:38.905688 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:01:38.913989 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 910/1852] 10.222.0.46 () {58 vars in 1121 bytes} [Thu Jun 23 16:01:38 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 16:01:38.918491 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61536eb-3bc5-41b9-a771-d13b98d9509c admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:01:38.937403 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61536eb-3bc5-41b9-a771-d13b98d9509c admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:01:38.944673 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d503fe13-d3af-4400-8840-e7259baf645d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:01:38.945733 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d503fe13-d3af-4400-8840-e7259baf645d admin admin] REQUEST_METHOD: `PUT` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:38.946038 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d503fe13-d3af-4400-8840-e7259baf645d admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:38.946978 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d503fe13-d3af-4400-8840-e7259baf645d admin admin] PATH_INFO: `/v3/projects/de455a645e284cfdb215092a29627cd3/users/b866fae02f5e4fd7a46b3eadd8cc6df0/roles/e2a5bbf3200a4d4091d72bc7ff78c34d` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:38.960319 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e384adb0-eabf-4787-8c10-52d42cb2449c None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:01:38.966478 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d503fe13-d3af-4400-8840-e7259baf645d admin admin] RBAC: Authorizing `identity:create_grant(project_id=de455a645e284cfdb215092a29627cd3, user_id=b866fae02f5e4fd7a46b3eadd8cc6df0, role_id=e2a5bbf3200a4d4091d72bc7ff78c34d)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:01:38.969312 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d503fe13-d3af-4400-8840-e7259baf645d admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:01:38.970528 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ad5b14-a31d-486d-a9c3-1f1686e1d4e1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce18eaaba3a6477684fff1fd98ec8b51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:01:38.971392 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ad5b14-a31d-486d-a9c3-1f1686e1d4e1 service cinder] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:38.971744 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ad5b14-a31d-486d-a9c3-1f1686e1d4e1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:38.972080 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ad5b14-a31d-486d-a9c3-1f1686e1d4e1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:38.985394 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ad5b14-a31d-486d-a9c3-1f1686e1d4e1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:01:38.987093 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ad5b14-a31d-486d-a9c3-1f1686e1d4e1 service cinder] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:01:38.992748 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 911/1853] 10.222.0.46 () {60 vars in 1482 bytes} [Thu Jun 23 16:01:38 2022] PUT /identity/v3/projects/de455a645e284cfdb215092a29627cd3/users/b866fae02f5e4fd7a46b3eadd8cc6df0/roles/e2a5bbf3200a4d4091d72bc7ff78c34d => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 16:01:39.004011 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 943/1854] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jun 23 16:01:38 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 16:01:39.004828 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe31c25-d4eb-4777-9bd4-e4707cdc1bbb None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:39.005122 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe31c25-d4eb-4777-9bd4-e4707cdc1bbb None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:39.005405 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe31c25-d4eb-4777-9bd4-e4707cdc1bbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:39.044838 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-dbe31c25-d4eb-4777-9bd4-e4707cdc1bbb None None] MFA Rules not processed for user `b866fae02f5e4fd7a46b3eadd8cc6df0`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:01:39.046294 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-dbe31c25-d4eb-4777-9bd4-e4707cdc1bbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:01:39.161311 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 912/1855] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:01:39 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 16:01:39.166961 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ad5b14-a31d-486d-a9c3-1f1686e1d4e1 service cinder] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:01:39.181960 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ad5b14-a31d-486d-a9c3-1f1686e1d4e1 service cinder] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:01:39.283861 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9508ae48-7b20-4b24-97ff-92376165e19d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:01:39.284599 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9508ae48-7b20-4b24-97ff-92376165e19d admin admin] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:39.284976 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9508ae48-7b20-4b24-97ff-92376165e19d admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:39.285291 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9508ae48-7b20-4b24-97ff-92376165e19d admin admin] PATH_INFO: `/v3/projects` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:39.286019 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9508ae48-7b20-4b24-97ff-92376165e19d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:01:39.288101 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9508ae48-7b20-4b24-97ff-92376165e19d admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:01:39.300516 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:01:39.328613 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 944/1856] 10.222.0.46 () {60 vars in 1152 bytes} [Thu Jun 23 16:01:39 2022] POST /identity/v3/projects => generated 379 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 16:01:39.332907 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe31c25-d4eb-4777-9bd4-e4707cdc1bbb None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:01:39.359382 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe31c25-d4eb-4777-9bd4-e4707cdc1bbb None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:01:39.429966 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba0cf6b-2591-415d-a9d1-2e15102595d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:01:39.430736 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba0cf6b-2591-415d-a9d1-2e15102595d7 admin admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:39.431010 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba0cf6b-2591-415d-a9d1-2e15102595d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:39.431356 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba0cf6b-2591-415d-a9d1-2e15102595d7 admin admin] PATH_INFO: `/v3/users` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:39.432061 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba0cf6b-2591-415d-a9d1-2e15102595d7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:01:39.434194 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba0cf6b-2591-415d-a9d1-2e15102595d7 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:01:39.467558 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 913/1857] 10.222.0.46 () {60 vars in 1143 bytes} [Thu Jun 23 16:01:39 2022] POST /identity/v3/users => generated 296 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 16:01:39.468732 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9508ae48-7b20-4b24-97ff-92376165e19d admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:01:39.485249 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9508ae48-7b20-4b24-97ff-92376165e19d admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:01:39.493250 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4089cbb9-0e46-4982-a0c8-7b06c894bbe4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:01:39.494465 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4089cbb9-0e46-4982-a0c8-7b06c894bbe4 admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:39.495010 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4089cbb9-0e46-4982-a0c8-7b06c894bbe4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:39.495485 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4089cbb9-0e46-4982-a0c8-7b06c894bbe4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:39.497024 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4089cbb9-0e46-4982-a0c8-7b06c894bbe4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:01:39.501040 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4089cbb9-0e46-4982-a0c8-7b06c894bbe4 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:01:39.528161 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 945/1858] 10.222.0.46 () {58 vars in 1121 bytes} [Thu Jun 23 16:01:39 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 16:01:39.530263 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba0cf6b-2591-415d-a9d1-2e15102595d7 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:01:39.545242 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4089cbb9-0e46-4982-a0c8-7b06c894bbe4 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:01:39.549922 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba0cf6b-2591-415d-a9d1-2e15102595d7 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:01:39.558197 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d842448a-572c-4afc-ba48-645b79bf8c08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:01:39.559713 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d842448a-572c-4afc-ba48-645b79bf8c08 admin admin] REQUEST_METHOD: `PUT` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:39.560217 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d842448a-572c-4afc-ba48-645b79bf8c08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:39.560715 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d842448a-572c-4afc-ba48-645b79bf8c08 admin admin] PATH_INFO: `/v3/projects/5cab70dd64344e64b0a91dcb6bea15ce/users/6e147681d62a4b718f2ba0b991070836/roles/812d3c567c52469cb2b35709fbdc857c` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:39.564047 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4089cbb9-0e46-4982-a0c8-7b06c894bbe4 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:01:39.572705 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7740cc19-810f-4fd9-b13b-36149c606b8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:01:39.573697 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7740cc19-810f-4fd9-b13b-36149c606b8b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:39.574212 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7740cc19-810f-4fd9-b13b-36149c606b8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:39.574669 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7740cc19-810f-4fd9-b13b-36149c606b8b admin admin] PATH_INFO: `/v3/users/c7019da576d64eb595cfddb70e5d2eb3` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:39.580447 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7740cc19-810f-4fd9-b13b-36149c606b8b admin admin] RBAC: Authorizing `identity:delete_user(user_id=c7019da576d64eb595cfddb70e5d2eb3)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:01:39.583169 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7740cc19-810f-4fd9-b13b-36149c606b8b admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:01:39.588110 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d842448a-572c-4afc-ba48-645b79bf8c08 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5cab70dd64344e64b0a91dcb6bea15ce, user_id=6e147681d62a4b718f2ba0b991070836, role_id=812d3c567c52469cb2b35709fbdc857c)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:01:39.591426 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d842448a-572c-4afc-ba48-645b79bf8c08 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:01:39.609197 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 914/1859] 10.222.0.46 () {60 vars in 1482 bytes} [Thu Jun 23 16:01:39 2022] PUT /identity/v3/projects/5cab70dd64344e64b0a91dcb6bea15ce/users/6e147681d62a4b718f2ba0b991070836/roles/812d3c567c52469cb2b35709fbdc857c => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 16:01:39.613675 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d842448a-572c-4afc-ba48-645b79bf8c08 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:01:39.682668 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:01:39.695407 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-7740cc19-810f-4fd9-b13b-36149c606b8b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c7019da576d64eb595cfddb70e5d2eb3'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:01:39.715837 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-7740cc19-810f-4fd9-b13b-36149c606b8b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c7019da576d64eb595cfddb70e5d2eb3'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:01:39.717152 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-7740cc19-810f-4fd9-b13b-36149c606b8b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c7019da576d64eb595cfddb70e5d2eb3'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:01:39.719663 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-7740cc19-810f-4fd9-b13b-36149c606b8b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c7019da576d64eb595cfddb70e5d2eb3'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:01:39.738590 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-7740cc19-810f-4fd9-b13b-36149c606b8b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c7019da576d64eb595cfddb70e5d2eb3'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:01:39.757138 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d842448a-572c-4afc-ba48-645b79bf8c08 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:01:39.769832 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fa2175-c477-41d1-a418-54f2249e24c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:01:39.771495 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fa2175-c477-41d1-a418-54f2249e24c0 admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:39.772044 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fa2175-c477-41d1-a418-54f2249e24c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:39.772525 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fa2175-c477-41d1-a418-54f2249e24c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:39.774338 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30fa2175-c477-41d1-a418-54f2249e24c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:01:39.777327 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30fa2175-c477-41d1-a418-54f2249e24c0 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:01:39.785008 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 946/1860] 10.222.0.46 () {58 vars in 1223 bytes} [Thu Jun 23 16:01:39 2022] DELETE /identity/v3/users/c7019da576d64eb595cfddb70e5d2eb3 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 16:01:39.794679 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 915/1861] 10.222.0.46 () {58 vars in 1121 bytes} [Thu Jun 23 16:01:39 2022] GET /identity/v3/roles => generated 1420 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 16:01:39.794961 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a77ef7-ccbc-49fc-82b7-518a2ae22006 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:39.795438 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a77ef7-ccbc-49fc-82b7-518a2ae22006 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:39.795798 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fa2175-c477-41d1-a418-54f2249e24c0 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:01:39.795953 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a77ef7-ccbc-49fc-82b7-518a2ae22006 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:39.797530 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-30fa2175-c477-41d1-a418-54f2249e24c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:01:39.816786 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fa2175-c477-41d1-a418-54f2249e24c0 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:01:39.825813 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-b2a77ef7-ccbc-49fc-82b7-518a2ae22006 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:01:39.828819 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-b2a77ef7-ccbc-49fc-82b7-518a2ae22006 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:01:39.902497 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52217e1c-76b9-4edd-bc18-494fb17d4188 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:01:39.903934 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52217e1c-76b9-4edd-bc18-494fb17d4188 admin admin] REQUEST_METHOD: `PUT` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:39.904327 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52217e1c-76b9-4edd-bc18-494fb17d4188 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:39.904709 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52217e1c-76b9-4edd-bc18-494fb17d4188 admin admin] PATH_INFO: `/v3/projects/5cab70dd64344e64b0a91dcb6bea15ce/users/6e147681d62a4b718f2ba0b991070836/roles/812d3c567c52469cb2b35709fbdc857c` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:39.907279 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52217e1c-76b9-4edd-bc18-494fb17d4188 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5cab70dd64344e64b0a91dcb6bea15ce, user_id=6e147681d62a4b718f2ba0b991070836, role_id=812d3c567c52469cb2b35709fbdc857c)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:01:39.910219 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52217e1c-76b9-4edd-bc18-494fb17d4188 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:01:39.927044 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 916/1862] 10.222.0.46 () {60 vars in 1482 bytes} [Thu Jun 23 16:01:39 2022] PUT /identity/v3/projects/5cab70dd64344e64b0a91dcb6bea15ce/users/6e147681d62a4b718f2ba0b991070836/roles/812d3c567c52469cb2b35709fbdc857c => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 16:01:39.938389 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e169b4f4-a390-44cb-9f22-14849c5e94b7 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:39.938867 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e169b4f4-a390-44cb-9f22-14849c5e94b7 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:39.939281 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e169b4f4-a390-44cb-9f22-14849c5e94b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:39.965875 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 947/1863] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:01:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 16:01:39.982331 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a77ef7-ccbc-49fc-82b7-518a2ae22006 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:01:39.985204 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-b2a77ef7-ccbc-49fc-82b7-518a2ae22006 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:01:39.997626 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-e169b4f4-a390-44cb-9f22-14849c5e94b7 None None] MFA Rules not processed for user `6e147681d62a4b718f2ba0b991070836`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:01:39.998958 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-e169b4f4-a390-44cb-9f22-14849c5e94b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:01:40.073201 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a77ef7-ccbc-49fc-82b7-518a2ae22006 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:01:40.081335 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:01:40.161759 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 917/1864] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:01:39 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 16:01:40.170674 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9738328-014c-4e0b-8f53-c9d9e512c65b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51784064951a4619ae09b9167fb0fd0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:01:40.171847 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9738328-014c-4e0b-8f53-c9d9e512c65b service neutron] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:40.172546 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9738328-014c-4e0b-8f53-c9d9e512c65b service neutron] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:40.174506 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e169b4f4-a390-44cb-9f22-14849c5e94b7 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:01:40.179928 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-e169b4f4-a390-44cb-9f22-14849c5e94b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:01:40.180199 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9738328-014c-4e0b-8f53-c9d9e512c65b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:40.268839 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9738328-014c-4e0b-8f53-c9d9e512c65b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:01:40.270380 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9738328-014c-4e0b-8f53-c9d9e512c65b service neutron] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:01:40.282135 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 948/1865] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jun 23 16:01:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 16:01:40.308435 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:01:40.331056 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e169b4f4-a390-44cb-9f22-14849c5e94b7 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:01:40.336058 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb408fef-8929-4507-a778-91d72d1d4f9c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce18eaaba3a6477684fff1fd98ec8b51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:01:40.338504 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb408fef-8929-4507-a778-91d72d1d4f9c service cinder] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:40.338889 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb408fef-8929-4507-a778-91d72d1d4f9c service cinder] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:40.339216 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb408fef-8929-4507-a778-91d72d1d4f9c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:40.351765 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb408fef-8929-4507-a778-91d72d1d4f9c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:01:40.354415 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb408fef-8929-4507-a778-91d72d1d4f9c service cinder] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:01:40.380093 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 918/1866] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jun 23 16:01:40 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 16:01:41.006306 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9738328-014c-4e0b-8f53-c9d9e512c65b service neutron] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:01:41.007961 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-f9738328-014c-4e0b-8f53-c9d9e512c65b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:01:41.031219 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9738328-014c-4e0b-8f53-c9d9e512c65b service neutron] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:01:41.039473 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb408fef-8929-4507-a778-91d72d1d4f9c service cinder] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:01:41.051707 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740b00ab-863b-440a-b76a-95c796a02379 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:01:41.055242 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb408fef-8929-4507-a778-91d72d1d4f9c service cinder] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:01:41.062188 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4742ba-a5d1-439f-97c1-e3b4a91e5781 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:01:41.063751 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4742ba-a5d1-439f-97c1-e3b4a91e5781 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:41.064467 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4742ba-a5d1-439f-97c1-e3b4a91e5781 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:41.064822 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4742ba-a5d1-439f-97c1-e3b4a91e5781 admin admin] PATH_INFO: `/v3/users/b866fae02f5e4fd7a46b3eadd8cc6df0` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:41.066663 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4742ba-a5d1-439f-97c1-e3b4a91e5781 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b866fae02f5e4fd7a46b3eadd8cc6df0)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:01:41.069129 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4742ba-a5d1-439f-97c1-e3b4a91e5781 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:01:41.072969 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740b00ab-863b-440a-b76a-95c796a02379 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:41.073520 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740b00ab-863b-440a-b76a-95c796a02379 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:41.074058 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740b00ab-863b-440a-b76a-95c796a02379 admin admin] PATH_INFO: `/v3/projects/26dfef1961b54ec798a413ef55c72333` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:41.077792 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740b00ab-863b-440a-b76a-95c796a02379 admin admin] RBAC: Authorizing `identity:delete_project(project_id=26dfef1961b54ec798a413ef55c72333)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:01:41.084426 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740b00ab-863b-440a-b76a-95c796a02379 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:01:41.183704 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-740b00ab-863b-440a-b76a-95c796a02379 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '26dfef1961b54ec798a413ef55c72333', 'request_id': 'req-740b00ab-863b-440a-b76a-95c796a02379'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:01:41.184155 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-dc4742ba-a5d1-439f-97c1-e3b4a91e5781 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b866fae02f5e4fd7a46b3eadd8cc6df0'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:01:41.185054 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-dc4742ba-a5d1-439f-97c1-e3b4a91e5781 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b866fae02f5e4fd7a46b3eadd8cc6df0'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:01:41.194791 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-740b00ab-863b-440a-b76a-95c796a02379 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '26dfef1961b54ec798a413ef55c72333', 'request_id': 'req-740b00ab-863b-440a-b76a-95c796a02379'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:01:41.204751 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-740b00ab-863b-440a-b76a-95c796a02379 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '26dfef1961b54ec798a413ef55c72333', 'request_id': 'req-740b00ab-863b-440a-b76a-95c796a02379'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:01:41.204990 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-dc4742ba-a5d1-439f-97c1-e3b4a91e5781 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b866fae02f5e4fd7a46b3eadd8cc6df0'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:01:41.219168 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-dc4742ba-a5d1-439f-97c1-e3b4a91e5781 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b866fae02f5e4fd7a46b3eadd8cc6df0'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:01:41.219918 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-dc4742ba-a5d1-439f-97c1-e3b4a91e5781 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b866fae02f5e4fd7a46b3eadd8cc6df0'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:01:41.235309 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-740b00ab-863b-440a-b76a-95c796a02379 admin admin] The token cache is being invalidate because project 26dfef1961b54ec798a413ef55c72333 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80153) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 16:01:41.236628 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-740b00ab-863b-440a-b76a-95c796a02379 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:01:41.237967 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-740b00ab-863b-440a-b76a-95c796a02379 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:01:41.240135 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 949/1867] 10.222.0.46 () {58 vars in 1232 bytes} [Thu Jun 23 16:01:41 2022] DELETE /identity/v3/projects/26dfef1961b54ec798a413ef55c72333 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 16:01:41.249300 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740b00ab-863b-440a-b76a-95c796a02379 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:01:41.254149 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-740b00ab-863b-440a-b76a-95c796a02379 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:01:41.269702 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 919/1868] 10.222.0.46 () {58 vars in 1223 bytes} [Thu Jun 23 16:01:41 2022] DELETE /identity/v3/users/b866fae02f5e4fd7a46b3eadd8cc6df0 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 16:01:41.280668 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a143b944-cff1-4f28-956b-d850fa706a2e None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:41.281054 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a143b944-cff1-4f28-956b-d850fa706a2e None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:41.281537 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a143b944-cff1-4f28-956b-d850fa706a2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:41.334270 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-a143b944-cff1-4f28-956b-d850fa706a2e None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:01:41.335632 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-a143b944-cff1-4f28-956b-d850fa706a2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:01:41.356260 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:01:41.404619 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740b00ab-863b-440a-b76a-95c796a02379 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:01:41.452145 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 920/1869] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:01:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 16:01:41.453623 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c5e71e-8e96-4e92-9ec0-b48135a0ef3e None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:41.453917 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c5e71e-8e96-4e92-9ec0-b48135a0ef3e None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:41.454201 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c5e71e-8e96-4e92-9ec0-b48135a0ef3e None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:41.460812 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 921/1870] 10.222.0.46 () {54 vars in 921 bytes} [Thu Jun 23 16:01:41 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 16:01:41.476358 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c5e71e-8e96-4e92-9ec0-b48135a0ef3e None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:01:41.478314 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-a8c5e71e-8e96-4e92-9ec0-b48135a0ef3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:01:41.544726 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae10d748-c90c-407d-8aa2-cc6f8c162db9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:01:41.546116 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae10d748-c90c-407d-8aa2-cc6f8c162db9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:41.546493 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae10d748-c90c-407d-8aa2-cc6f8c162db9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:41.546922 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae10d748-c90c-407d-8aa2-cc6f8c162db9 admin admin] PATH_INFO: `/v3/users/91d4ac439fd44121b44498d66df8d00e` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:41.548425 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae10d748-c90c-407d-8aa2-cc6f8c162db9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=91d4ac439fd44121b44498d66df8d00e)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:01:41.553622 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae10d748-c90c-407d-8aa2-cc6f8c162db9 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:01:41.629402 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c5e71e-8e96-4e92-9ec0-b48135a0ef3e None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:01:41.646940 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ace8c3-e01c-461a-bd7f-7be4dcaa315a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51784064951a4619ae09b9167fb0fd0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:01:41.646940 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ace8c3-e01c-461a-bd7f-7be4dcaa315a service neutron] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:41.646940 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ace8c3-e01c-461a-bd7f-7be4dcaa315a service neutron] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:41.646940 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ace8c3-e01c-461a-bd7f-7be4dcaa315a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:41.671658 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ace8c3-e01c-461a-bd7f-7be4dcaa315a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:01:41.673238 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ace8c3-e01c-461a-bd7f-7be4dcaa315a service neutron] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:01:41.685222 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-ae10d748-c90c-407d-8aa2-cc6f8c162db9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '91d4ac439fd44121b44498d66df8d00e'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:01:41.701132 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-ae10d748-c90c-407d-8aa2-cc6f8c162db9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '91d4ac439fd44121b44498d66df8d00e'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:01:41.702082 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 922/1871] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jun 23 16:01:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 16:01:41.703556 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-ae10d748-c90c-407d-8aa2-cc6f8c162db9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '91d4ac439fd44121b44498d66df8d00e'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:01:41.704914 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45288bc4-dc39-4b0a-a082-57f92aae3f9b None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:41.705476 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-ae10d748-c90c-407d-8aa2-cc6f8c162db9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '91d4ac439fd44121b44498d66df8d00e'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:01:41.705873 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45288bc4-dc39-4b0a-a082-57f92aae3f9b None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:41.715503 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45288bc4-dc39-4b0a-a082-57f92aae3f9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:41.733516 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:01:41.742798 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-ae10d748-c90c-407d-8aa2-cc6f8c162db9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '91d4ac439fd44121b44498d66df8d00e'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:01:41.755228 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-45288bc4-dc39-4b0a-a082-57f92aae3f9b None None] MFA Rules not processed for user `705115d19bab424d894bea8b1b48a7f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:01:41.756562 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-45288bc4-dc39-4b0a-a082-57f92aae3f9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:01:41.785882 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 950/1872] 10.222.0.46 () {58 vars in 1223 bytes} [Thu Jun 23 16:01:41 2022] DELETE /identity/v3/users/91d4ac439fd44121b44498d66df8d00e => generated 0 bytes in 539 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 16:01:41.867501 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 923/1873] 10.222.0.46 () {58 vars in 1018 bytes} [Thu Jun 23 16:01:41 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 16:01:41.886528 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae10d748-c90c-407d-8aa2-cc6f8c162db9 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:01:41.888220 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-ae10d748-c90c-407d-8aa2-cc6f8c162db9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:01:42.061236 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae10d748-c90c-407d-8aa2-cc6f8c162db9 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:01:42.097190 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985b4a58-6162-406c-b0a8-73e0c5872f68 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '705115d19bab424d894bea8b1b48a7f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:01:42.097871 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985b4a58-6162-406c-b0a8-73e0c5872f68 service nova] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:42.098159 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985b4a58-6162-406c-b0a8-73e0c5872f68 service nova] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:42.098438 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985b4a58-6162-406c-b0a8-73e0c5872f68 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:42.109502 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:01:42.121490 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985b4a58-6162-406c-b0a8-73e0c5872f68 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:01:42.123885 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985b4a58-6162-406c-b0a8-73e0c5872f68 service nova] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:01:42.139563 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45288bc4-dc39-4b0a-a082-57f92aae3f9b None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:01:42.141130 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-45288bc4-dc39-4b0a-a082-57f92aae3f9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:01:42.157002 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 951/1874] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jun 23 16:01:41 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 271 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 16:01:42.179151 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45288bc4-dc39-4b0a-a082-57f92aae3f9b None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:01:42.263476 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985b4a58-6162-406c-b0a8-73e0c5872f68 service nova] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:01:42.271650 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-985b4a58-6162-406c-b0a8-73e0c5872f68 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:01:42.293577 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06c17ea-15aa-42dd-bce3-1b40b3253ebd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:01:42.294833 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06c17ea-15aa-42dd-bce3-1b40b3253ebd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:42.295190 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06c17ea-15aa-42dd-bce3-1b40b3253ebd admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:42.295495 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06c17ea-15aa-42dd-bce3-1b40b3253ebd admin admin] PATH_INFO: `/v3/projects/de455a645e284cfdb215092a29627cd3` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:42.297039 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06c17ea-15aa-42dd-bce3-1b40b3253ebd admin admin] RBAC: Authorizing `identity:delete_project(project_id=de455a645e284cfdb215092a29627cd3)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:01:42.306730 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06c17ea-15aa-42dd-bce3-1b40b3253ebd admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:01:42.466113 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-f06c17ea-15aa-42dd-bce3-1b40b3253ebd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'de455a645e284cfdb215092a29627cd3', 'request_id': 'req-f06c17ea-15aa-42dd-bce3-1b40b3253ebd'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:01:42.478182 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-f06c17ea-15aa-42dd-bce3-1b40b3253ebd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'de455a645e284cfdb215092a29627cd3', 'request_id': 'req-f06c17ea-15aa-42dd-bce3-1b40b3253ebd'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:01:42.486593 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985b4a58-6162-406c-b0a8-73e0c5872f68 service nova] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:01:42.490495 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-f06c17ea-15aa-42dd-bce3-1b40b3253ebd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'de455a645e284cfdb215092a29627cd3', 'request_id': 'req-f06c17ea-15aa-42dd-bce3-1b40b3253ebd'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:01:42.505682 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-f06c17ea-15aa-42dd-bce3-1b40b3253ebd admin admin] The token cache is being invalidate because project de455a645e284cfdb215092a29627cd3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80154) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 16:01:42.514122 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-f06c17ea-15aa-42dd-bce3-1b40b3253ebd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:01:42.515030 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-f06c17ea-15aa-42dd-bce3-1b40b3253ebd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:01:42.522797 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 924/1875] 10.222.0.46 () {58 vars in 1232 bytes} [Thu Jun 23 16:01:42 2022] DELETE /identity/v3/projects/de455a645e284cfdb215092a29627cd3 => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 16:01:42.523833 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06c17ea-15aa-42dd-bce3-1b40b3253ebd admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:01:42.525807 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-f06c17ea-15aa-42dd-bce3-1b40b3253ebd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:01:42.613892 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:01:42.648364 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5027937-315e-4dbc-9581-bc4407b8c9a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:01:42.649998 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5027937-315e-4dbc-9581-bc4407b8c9a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:42.651140 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5027937-315e-4dbc-9581-bc4407b8c9a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:42.651754 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5027937-315e-4dbc-9581-bc4407b8c9a1 admin admin] PATH_INFO: `/v3/projects/05d4d1c45d234e3bbc735192eec32bed` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:42.659019 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5027937-315e-4dbc-9581-bc4407b8c9a1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=05d4d1c45d234e3bbc735192eec32bed)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:01:42.664593 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5027937-315e-4dbc-9581-bc4407b8c9a1 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:01:42.679092 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06c17ea-15aa-42dd-bce3-1b40b3253ebd admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:01:42.693065 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e97023-8180-422a-b321-851df4df5ead admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:01:42.695010 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e97023-8180-422a-b321-851df4df5ead admin admin] REQUEST_METHOD: `DELETE` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:42.695370 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e97023-8180-422a-b321-851df4df5ead admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:42.695858 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e97023-8180-422a-b321-851df4df5ead admin admin] PATH_INFO: `/v3/users/6e147681d62a4b718f2ba0b991070836` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:42.697657 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e97023-8180-422a-b321-851df4df5ead admin admin] RBAC: Authorizing `identity:delete_user(user_id=6e147681d62a4b718f2ba0b991070836)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:01:42.699911 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e97023-8180-422a-b321-851df4df5ead admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:01:42.795304 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-b5027937-315e-4dbc-9581-bc4407b8c9a1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '05d4d1c45d234e3bbc735192eec32bed', 'request_id': 'req-b5027937-315e-4dbc-9581-bc4407b8c9a1'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:01:42.795820 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-03e97023-8180-422a-b321-851df4df5ead admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6e147681d62a4b718f2ba0b991070836'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:01:42.796553 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-03e97023-8180-422a-b321-851df4df5ead admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6e147681d62a4b718f2ba0b991070836'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:01:42.805660 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-03e97023-8180-422a-b321-851df4df5ead admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6e147681d62a4b718f2ba0b991070836'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:01:42.811461 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-b5027937-315e-4dbc-9581-bc4407b8c9a1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '05d4d1c45d234e3bbc735192eec32bed', 'request_id': 'req-b5027937-315e-4dbc-9581-bc4407b8c9a1'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:01:42.818665 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-03e97023-8180-422a-b321-851df4df5ead admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6e147681d62a4b718f2ba0b991070836'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:01:42.819360 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-03e97023-8180-422a-b321-851df4df5ead admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6e147681d62a4b718f2ba0b991070836'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:01:42.832910 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-b5027937-315e-4dbc-9581-bc4407b8c9a1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '05d4d1c45d234e3bbc735192eec32bed', 'request_id': 'req-b5027937-315e-4dbc-9581-bc4407b8c9a1'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:01:42.877910 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-b5027937-315e-4dbc-9581-bc4407b8c9a1 admin admin] The token cache is being invalidate because project 05d4d1c45d234e3bbc735192eec32bed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80153) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 16:01:42.878527 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-b5027937-315e-4dbc-9581-bc4407b8c9a1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:01:42.879755 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-b5027937-315e-4dbc-9581-bc4407b8c9a1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:01:42.888987 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 952/1876] 10.222.0.46 () {58 vars in 1232 bytes} [Thu Jun 23 16:01:42 2022] DELETE /identity/v3/projects/05d4d1c45d234e3bbc735192eec32bed => generated 0 bytes in 626 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 16:01:42.898619 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ad05f8-3904-41d8-9d04-7d1b7d76fc2b None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:42.901355 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ad05f8-3904-41d8-9d04-7d1b7d76fc2b None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:42.902427 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ad05f8-3904-41d8-9d04-7d1b7d76fc2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:42.923223 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 925/1877] 10.222.0.46 () {58 vars in 1223 bytes} [Thu Jun 23 16:01:42 2022] DELETE /identity/v3/users/6e147681d62a4b718f2ba0b991070836 => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 16:01:43.001522 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-61ad05f8-3904-41d8-9d04-7d1b7d76fc2b None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:01:43.005370 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-61ad05f8-3904-41d8-9d04-7d1b7d76fc2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:01:43.103209 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:01:43.182501 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 953/1878] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:01:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 16:01:43.185432 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f982cb-b476-4118-b00d-2ec666f5a3e5 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:43.186588 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f982cb-b476-4118-b00d-2ec666f5a3e5 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:43.187359 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f982cb-b476-4118-b00d-2ec666f5a3e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:43.223239 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-04f982cb-b476-4118-b00d-2ec666f5a3e5 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:01:43.225008 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-04f982cb-b476-4118-b00d-2ec666f5a3e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:01:43.240288 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 926/1879] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:01:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 16:01:43.246571 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ad05f8-3904-41d8-9d04-7d1b7d76fc2b None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:01:43.257392 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ad05f8-3904-41d8-9d04-7d1b7d76fc2b None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:01:43.267800 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f415719-41d8-4938-a772-0a4b6d4d0073 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:01:43.268401 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f415719-41d8-4938-a772-0a4b6d4d0073 admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:43.268686 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f415719-41d8-4938-a772-0a4b6d4d0073 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:43.268963 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f415719-41d8-4938-a772-0a4b6d4d0073 admin admin] PATH_INFO: `/v3/domains` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:43.269728 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f415719-41d8-4938-a772-0a4b6d4d0073 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 16:01:43.270043 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f415719-41d8-4938-a772-0a4b6d4d0073 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:01:43.271778 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f415719-41d8-4938-a772-0a4b6d4d0073 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:01:43.287402 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 954/1880] 10.222.0.46 () {58 vars in 1152 bytes} [Thu Jun 23 16:01:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 16:01:43.295333 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f982cb-b476-4118-b00d-2ec666f5a3e5 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:01:43.308118 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f982cb-b476-4118-b00d-2ec666f5a3e5 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:01:43.322162 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1456f7-4208-48f2-8592-7e44a32ec81b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:01:43.323576 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1456f7-4208-48f2-8592-7e44a32ec81b admin admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:43.323947 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1456f7-4208-48f2-8592-7e44a32ec81b admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:43.324145 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1456f7-4208-48f2-8592-7e44a32ec81b admin admin] PATH_INFO: `/v3/projects` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:43.326514 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1456f7-4208-48f2-8592-7e44a32ec81b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:01:43.329437 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1456f7-4208-48f2-8592-7e44a32ec81b admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:01:43.387758 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 927/1881] 10.222.0.46 () {60 vars in 1152 bytes} [Thu Jun 23 16:01:43 2022] POST /identity/v3/projects => generated 363 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 16:01:43.392645 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f415719-41d8-4938-a772-0a4b6d4d0073 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:01:43.433363 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f415719-41d8-4938-a772-0a4b6d4d0073 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:01:43.557931 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0f7c12-5c84-4a38-b417-d4c5e9ba1212 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:01:43.559105 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0f7c12-5c84-4a38-b417-d4c5e9ba1212 admin admin] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:43.567298 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0f7c12-5c84-4a38-b417-d4c5e9ba1212 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:43.567724 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0f7c12-5c84-4a38-b417-d4c5e9ba1212 admin admin] PATH_INFO: `/v3/users` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:43.569493 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0f7c12-5c84-4a38-b417-d4c5e9ba1212 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:01:43.572283 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1456f7-4208-48f2-8592-7e44a32ec81b admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:01:43.573128 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0f7c12-5c84-4a38-b417-d4c5e9ba1212 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:01:43.574853 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-fa1456f7-4208-48f2-8592-7e44a32ec81b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:01:43.615437 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1456f7-4208-48f2-8592-7e44a32ec81b admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:01:43.625302 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 955/1882] 10.222.0.46 () {60 vars in 1143 bytes} [Thu Jun 23 16:01:43 2022] POST /identity/v3/users => generated 282 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 16:01:43.627179 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3d81ba-e4eb-4316-bc67-88a749869ec8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:01:43.628104 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3d81ba-e4eb-4316-bc67-88a749869ec8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:43.628531 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3d81ba-e4eb-4316-bc67-88a749869ec8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:43.630085 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0f7c12-5c84-4a38-b417-d4c5e9ba1212 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:01:43.630786 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3d81ba-e4eb-4316-bc67-88a749869ec8 admin admin] PATH_INFO: `/v3/projects/5cab70dd64344e64b0a91dcb6bea15ce` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:43.637598 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3d81ba-e4eb-4316-bc67-88a749869ec8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5cab70dd64344e64b0a91dcb6bea15ce)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:01:43.641068 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3d81ba-e4eb-4316-bc67-88a749869ec8 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:01:43.658237 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0f7c12-5c84-4a38-b417-d4c5e9ba1212 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:01:43.673352 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddafdeb4-8eee-4a68-8fb2-8f7db054139a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:01:43.678889 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddafdeb4-8eee-4a68-8fb2-8f7db054139a admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:43.679314 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddafdeb4-8eee-4a68-8fb2-8f7db054139a admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:43.682475 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddafdeb4-8eee-4a68-8fb2-8f7db054139a admin admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:43.685813 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddafdeb4-8eee-4a68-8fb2-8f7db054139a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:01:43.688156 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddafdeb4-8eee-4a68-8fb2-8f7db054139a admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:01:43.714746 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 956/1883] 10.222.0.46 () {58 vars in 1121 bytes} [Thu Jun 23 16:01:43 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 16:01:43.718336 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddafdeb4-8eee-4a68-8fb2-8f7db054139a admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:01:43.743991 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddafdeb4-8eee-4a68-8fb2-8f7db054139a admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:01:43.750219 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbec740-e5fb-4c57-a8a7-9841aa5ab661 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:01:43.751262 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbec740-e5fb-4c57-a8a7-9841aa5ab661 admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:43.751701 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbec740-e5fb-4c57-a8a7-9841aa5ab661 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:43.752232 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbec740-e5fb-4c57-a8a7-9841aa5ab661 admin admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:43.753147 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbec740-e5fb-4c57-a8a7-9841aa5ab661 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:01:43.756061 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbec740-e5fb-4c57-a8a7-9841aa5ab661 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:01:43.770449 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 957/1884] 10.222.0.46 () {58 vars in 1121 bytes} [Thu Jun 23 16:01:43 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 16:01:43.777042 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbec740-e5fb-4c57-a8a7-9841aa5ab661 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:01:43.786454 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-3f3d81ba-e4eb-4316-bc67-88a749869ec8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5cab70dd64344e64b0a91dcb6bea15ce', 'request_id': 'req-3f3d81ba-e4eb-4316-bc67-88a749869ec8'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:01:43.798611 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-3f3d81ba-e4eb-4316-bc67-88a749869ec8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5cab70dd64344e64b0a91dcb6bea15ce', 'request_id': 'req-3f3d81ba-e4eb-4316-bc67-88a749869ec8'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:01:43.816200 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-3f3d81ba-e4eb-4316-bc67-88a749869ec8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5cab70dd64344e64b0a91dcb6bea15ce', 'request_id': 'req-3f3d81ba-e4eb-4316-bc67-88a749869ec8'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:01:43.817259 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbec740-e5fb-4c57-a8a7-9841aa5ab661 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:01:43.831525 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=80,keystone:DELETE=17,keystone:INSERT=3 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:01:43.841869 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-3f3d81ba-e4eb-4316-bc67-88a749869ec8 admin admin] The token cache is being invalidate because project 5cab70dd64344e64b0a91dcb6bea15ce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80154) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 16:01:43.841869 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-3f3d81ba-e4eb-4316-bc67-88a749869ec8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:01:43.843288 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-3f3d81ba-e4eb-4316-bc67-88a749869ec8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:01:43.854496 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 928/1885] 10.222.0.46 () {58 vars in 1232 bytes} [Thu Jun 23 16:01:43 2022] DELETE /identity/v3/projects/5cab70dd64344e64b0a91dcb6bea15ce => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 16:01:43.861412 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac18e486-6472-403a-966f-386212ab5afc None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:43.862294 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac18e486-6472-403a-966f-386212ab5afc None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:43.862754 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac18e486-6472-403a-966f-386212ab5afc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:43.892706 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-ac18e486-6472-403a-966f-386212ab5afc None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:01:43.894043 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-ac18e486-6472-403a-966f-386212ab5afc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:01:43.962960 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f94898b-3658-4b15-8da1-d4731c4c0111 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:01:43.964141 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f94898b-3658-4b15-8da1-d4731c4c0111 admin admin] REQUEST_METHOD: `PUT` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:43.964457 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f94898b-3658-4b15-8da1-d4731c4c0111 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:43.964738 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f94898b-3658-4b15-8da1-d4731c4c0111 admin admin] PATH_INFO: `/v3/projects/0a416742729e4eaa95846ecf5a61c677/users/8eef5f1c93bb44efa0af8597bb28d648/roles/e2a5bbf3200a4d4091d72bc7ff78c34d` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:43.989374 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f94898b-3658-4b15-8da1-d4731c4c0111 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a416742729e4eaa95846ecf5a61c677, user_id=8eef5f1c93bb44efa0af8597bb28d648, role_id=e2a5bbf3200a4d4091d72bc7ff78c34d)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:01:43.992731 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f94898b-3658-4b15-8da1-d4731c4c0111 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:01:44.014406 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 958/1886] 10.222.0.46 () {60 vars in 1482 bytes} [Thu Jun 23 16:01:43 2022] PUT /identity/v3/projects/0a416742729e4eaa95846ecf5a61c677/users/8eef5f1c93bb44efa0af8597bb28d648/roles/e2a5bbf3200a4d4091d72bc7ff78c34d => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 16:01:44.033495 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed87b232-5afb-48b3-b156-dcd516636953 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:44.034388 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed87b232-5afb-48b3-b156-dcd516636953 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:44.035381 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed87b232-5afb-48b3-b156-dcd516636953 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:44.071683 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:01:44.106732 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-ed87b232-5afb-48b3-b156-dcd516636953 None None] MFA Rules not processed for user `8eef5f1c93bb44efa0af8597bb28d648`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:01:44.109234 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-ed87b232-5afb-48b3-b156-dcd516636953 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:01:44.121366 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 929/1887] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:01:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 16:01:44.125401 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06703610-472b-4f0f-a7ec-6bb45c72b3c2 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:44.125941 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06703610-472b-4f0f-a7ec-6bb45c72b3c2 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:44.126437 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06703610-472b-4f0f-a7ec-6bb45c72b3c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:44.185705 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-06703610-472b-4f0f-a7ec-6bb45c72b3c2 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:01:44.189690 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-06703610-472b-4f0f-a7ec-6bb45c72b3c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:01:44.293691 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 930/1888] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:01:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 16:01:44.298210 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06703610-472b-4f0f-a7ec-6bb45c72b3c2 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:01:44.306908 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 959/1889] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:01:44 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 275 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 16:01:44.311312 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06703610-472b-4f0f-a7ec-6bb45c72b3c2 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:01:44.317902 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4dc0dd-ba3c-4569-a1d6-0a08494fb664 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:01:44.317902 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4dc0dd-ba3c-4569-a1d6-0a08494fb664 admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:44.318293 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4dc0dd-ba3c-4569-a1d6-0a08494fb664 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:44.318472 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4dc0dd-ba3c-4569-a1d6-0a08494fb664 admin admin] PATH_INFO: `/v3/domains` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:44.319153 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4dc0dd-ba3c-4569-a1d6-0a08494fb664 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 16:01:44.319464 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4dc0dd-ba3c-4569-a1d6-0a08494fb664 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:01:44.321407 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4dc0dd-ba3c-4569-a1d6-0a08494fb664 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:01:44.330325 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed87b232-5afb-48b3-b156-dcd516636953 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:01:44.335275 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-ed87b232-5afb-48b3-b156-dcd516636953 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:01:44.338212 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 931/1890] 10.222.0.46 () {58 vars in 1152 bytes} [Thu Jun 23 16:01:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 16:01:44.343763 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4dc0dd-ba3c-4569-a1d6-0a08494fb664 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:01:44.361676 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4dc0dd-ba3c-4569-a1d6-0a08494fb664 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:01:44.369614 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db539491-ef24-42e4-85a7-5091b0172f3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:01:44.370538 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db539491-ef24-42e4-85a7-5091b0172f3e admin admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:44.372170 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db539491-ef24-42e4-85a7-5091b0172f3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:44.372500 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db539491-ef24-42e4-85a7-5091b0172f3e admin admin] PATH_INFO: `/v3/projects` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:44.373473 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db539491-ef24-42e4-85a7-5091b0172f3e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:01:44.375956 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db539491-ef24-42e4-85a7-5091b0172f3e admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:01:44.386154 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed87b232-5afb-48b3-b156-dcd516636953 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:01:44.398498 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d613df-726a-48eb-8b94-ebafdf98e869 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:01:44.402297 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d613df-726a-48eb-8b94-ebafdf98e869 admin admin] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:44.403423 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d613df-726a-48eb-8b94-ebafdf98e869 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:44.403564 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d613df-726a-48eb-8b94-ebafdf98e869 admin admin] PATH_INFO: `/v3/projects` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:44.405082 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d613df-726a-48eb-8b94-ebafdf98e869 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:01:44.408839 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d613df-726a-48eb-8b94-ebafdf98e869 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:01:44.430671 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 932/1891] 10.222.0.46 () {60 vars in 1152 bytes} [Thu Jun 23 16:01:44 2022] POST /identity/v3/projects => generated 373 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 16:01:44.434219 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db539491-ef24-42e4-85a7-5091b0172f3e admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:01:44.476999 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 960/1892] 10.222.0.46 () {60 vars in 1152 bytes} [Thu Jun 23 16:01:44 2022] POST /identity/v3/projects => generated 365 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 16:01:44.483994 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d613df-726a-48eb-8b94-ebafdf98e869 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:01:44.642346 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db539491-ef24-42e4-85a7-5091b0172f3e admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:01:44.667021 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:01:44.804658 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d613df-726a-48eb-8b94-ebafdf98e869 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:01:44.818876 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045d1f75-e129-477c-81fd-39b259d31b70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:01:44.820465 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045d1f75-e129-477c-81fd-39b259d31b70 admin admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:44.821598 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045d1f75-e129-477c-81fd-39b259d31b70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:44.822834 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045d1f75-e129-477c-81fd-39b259d31b70 admin admin] PATH_INFO: `/v3/users` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:44.825243 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045d1f75-e129-477c-81fd-39b259d31b70 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:01:44.827261 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af6084f-839a-40e2-9824-0a90700143f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:01:44.830760 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045d1f75-e129-477c-81fd-39b259d31b70 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:01:44.843820 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af6084f-839a-40e2-9824-0a90700143f7 admin admin] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:44.845037 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af6084f-839a-40e2-9824-0a90700143f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:44.846015 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af6084f-839a-40e2-9824-0a90700143f7 admin admin] PATH_INFO: `/v3/users` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:44.848335 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af6084f-839a-40e2-9824-0a90700143f7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:01:44.852324 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af6084f-839a-40e2-9824-0a90700143f7 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:01:44.895751 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 933/1893] 10.222.0.46 () {60 vars in 1143 bytes} [Thu Jun 23 16:01:44 2022] POST /identity/v3/users => generated 287 bytes in 463 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 16:01:44.897388 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045d1f75-e129-477c-81fd-39b259d31b70 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:01:44.906878 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 961/1894] 10.222.0.46 () {60 vars in 1143 bytes} [Thu Jun 23 16:01:44 2022] POST /identity/v3/users => generated 289 bytes in 420 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 16:01:44.909116 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af6084f-839a-40e2-9824-0a90700143f7 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:01:44.919684 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045d1f75-e129-477c-81fd-39b259d31b70 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:01:44.926347 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af6084f-839a-40e2-9824-0a90700143f7 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:01:44.930288 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04a4df6-a8cc-4535-a02f-964dc9aa53a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:01:44.931532 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04a4df6-a8cc-4535-a02f-964dc9aa53a5 admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:44.932033 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04a4df6-a8cc-4535-a02f-964dc9aa53a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:44.932527 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04a4df6-a8cc-4535-a02f-964dc9aa53a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:44.934361 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04a4df6-a8cc-4535-a02f-964dc9aa53a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:01:44.936216 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95a2f23-8ab8-4209-ba6d-947d4c57c1fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:01:44.936962 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95a2f23-8ab8-4209-ba6d-947d4c57c1fe admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:44.937327 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04a4df6-a8cc-4535-a02f-964dc9aa53a5 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:01:44.937794 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95a2f23-8ab8-4209-ba6d-947d4c57c1fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:44.938395 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95a2f23-8ab8-4209-ba6d-947d4c57c1fe admin admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:44.941317 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95a2f23-8ab8-4209-ba6d-947d4c57c1fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:01:44.945288 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95a2f23-8ab8-4209-ba6d-947d4c57c1fe admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:01:44.962945 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 934/1895] 10.222.0.46 () {58 vars in 1121 bytes} [Thu Jun 23 16:01:44 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 16:01:44.968770 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04a4df6-a8cc-4535-a02f-964dc9aa53a5 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:01:44.975197 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 962/1896] 10.222.0.46 () {58 vars in 1121 bytes} [Thu Jun 23 16:01:44 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 16:01:44.988330 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95a2f23-8ab8-4209-ba6d-947d4c57c1fe admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:01:44.994949 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04a4df6-a8cc-4535-a02f-964dc9aa53a5 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:01:45.004841 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a1379d-cbfe-4521-9706-b878ed99dccd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:01:45.006445 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a1379d-cbfe-4521-9706-b878ed99dccd admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:45.007131 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a1379d-cbfe-4521-9706-b878ed99dccd admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:45.007970 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a1379d-cbfe-4521-9706-b878ed99dccd admin admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:45.010779 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a1379d-cbfe-4521-9706-b878ed99dccd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:01:45.014032 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a1379d-cbfe-4521-9706-b878ed99dccd admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:01:45.014255 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95a2f23-8ab8-4209-ba6d-947d4c57c1fe admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:01:45.029383 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521f7c87-7fad-4ad0-af99-c57776b40cf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:01:45.032889 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521f7c87-7fad-4ad0-af99-c57776b40cf4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:45.033427 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521f7c87-7fad-4ad0-af99-c57776b40cf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:45.033938 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521f7c87-7fad-4ad0-af99-c57776b40cf4 admin admin] PATH_INFO: `/v3/projects/f444fc15e9b14ff389c7ca73afabf3e1/users/59199bbe764d4f0bafaccc015ad618f1/roles/812d3c567c52469cb2b35709fbdc857c` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:45.045043 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 935/1897] 10.222.0.46 () {58 vars in 1121 bytes} [Thu Jun 23 16:01:44 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 16:01:45.051001 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a1379d-cbfe-4521-9706-b878ed99dccd admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:01:45.071979 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a1379d-cbfe-4521-9706-b878ed99dccd admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:01:45.077307 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521f7c87-7fad-4ad0-af99-c57776b40cf4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f444fc15e9b14ff389c7ca73afabf3e1, user_id=59199bbe764d4f0bafaccc015ad618f1, role_id=812d3c567c52469cb2b35709fbdc857c)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:01:45.080365 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521f7c87-7fad-4ad0-af99-c57776b40cf4 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:01:45.084979 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e696074-f150-4278-aa38-25b4ab8db290 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:01:45.086985 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e696074-f150-4278-aa38-25b4ab8db290 admin admin] REQUEST_METHOD: `PUT` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:45.087527 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e696074-f150-4278-aa38-25b4ab8db290 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:45.088557 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e696074-f150-4278-aa38-25b4ab8db290 admin admin] PATH_INFO: `/v3/projects/ed4025121c444d1f86fdcb5d4e7b03e2/users/ea2f6774a63e45e9acc1f5eb46b0850d/roles/e2a5bbf3200a4d4091d72bc7ff78c34d` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:45.104143 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 963/1898] 10.222.0.46 () {60 vars in 1482 bytes} [Thu Jun 23 16:01:44 2022] PUT /identity/v3/projects/f444fc15e9b14ff389c7ca73afabf3e1/users/59199bbe764d4f0bafaccc015ad618f1/roles/812d3c567c52469cb2b35709fbdc857c => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 16:01:45.109578 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521f7c87-7fad-4ad0-af99-c57776b40cf4 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:01:45.131103 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e696074-f150-4278-aa38-25b4ab8db290 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed4025121c444d1f86fdcb5d4e7b03e2, user_id=ea2f6774a63e45e9acc1f5eb46b0850d, role_id=e2a5bbf3200a4d4091d72bc7ff78c34d)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:01:45.142000 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e696074-f150-4278-aa38-25b4ab8db290 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:01:45.190915 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 936/1899] 10.222.0.46 () {60 vars in 1482 bytes} [Thu Jun 23 16:01:45 2022] PUT /identity/v3/projects/ed4025121c444d1f86fdcb5d4e7b03e2/users/ea2f6774a63e45e9acc1f5eb46b0850d/roles/e2a5bbf3200a4d4091d72bc7ff78c34d => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 16:01:45.195806 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:01:45.216255 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52e7922-f9bc-4f2c-88eb-020342461931 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:45.217041 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52e7922-f9bc-4f2c-88eb-020342461931 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:45.218183 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52e7922-f9bc-4f2c-88eb-020342461931 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:45.297876 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-c52e7922-f9bc-4f2c-88eb-020342461931 None None] MFA Rules not processed for user `ea2f6774a63e45e9acc1f5eb46b0850d`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:01:45.300034 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-c52e7922-f9bc-4f2c-88eb-020342461931 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:01:45.336393 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521f7c87-7fad-4ad0-af99-c57776b40cf4 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:01:45.553250 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b899ead2-4923-4991-a9bd-598d00a93733 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:01:45.554842 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b899ead2-4923-4991-a9bd-598d00a93733 admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:45.562877 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b899ead2-4923-4991-a9bd-598d00a93733 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:45.563974 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b899ead2-4923-4991-a9bd-598d00a93733 admin admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:45.567525 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b899ead2-4923-4991-a9bd-598d00a93733 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:01:45.577972 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 937/1900] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:01:45 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 364 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 16:01:45.581003 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b899ead2-4923-4991-a9bd-598d00a93733 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:01:45.604732 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52e7922-f9bc-4f2c-88eb-020342461931 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:01:45.642941 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52e7922-f9bc-4f2c-88eb-020342461931 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:01:45.664397 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa251df3-3889-46ef-81ce-de5261fcc144 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:01:45.671525 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa251df3-3889-46ef-81ce-de5261fcc144 admin admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:45.672030 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa251df3-3889-46ef-81ce-de5261fcc144 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:45.674820 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa251df3-3889-46ef-81ce-de5261fcc144 admin admin] PATH_INFO: `/v3/projects` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:45.679966 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 964/1901] 10.222.0.46 () {58 vars in 1121 bytes} [Thu Jun 23 16:01:45 2022] GET /identity/v3/roles => generated 1420 bytes in 571 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 16:01:45.680311 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa251df3-3889-46ef-81ce-de5261fcc144 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:01:45.687130 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b899ead2-4923-4991-a9bd-598d00a93733 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:01:45.690485 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa251df3-3889-46ef-81ce-de5261fcc144 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:01:45.735605 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b899ead2-4923-4991-a9bd-598d00a93733 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:01:45.766442 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b661657d-4562-41c4-a808-f29c1809bccb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:01:45.770237 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b661657d-4562-41c4-a808-f29c1809bccb admin admin] REQUEST_METHOD: `PUT` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:45.773497 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b661657d-4562-41c4-a808-f29c1809bccb admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:45.776317 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b661657d-4562-41c4-a808-f29c1809bccb admin admin] PATH_INFO: `/v3/projects/f444fc15e9b14ff389c7ca73afabf3e1/users/59199bbe764d4f0bafaccc015ad618f1/roles/812d3c567c52469cb2b35709fbdc857c` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:45.789512 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b661657d-4562-41c4-a808-f29c1809bccb admin admin] RBAC: Authorizing `identity:create_grant(project_id=f444fc15e9b14ff389c7ca73afabf3e1, user_id=59199bbe764d4f0bafaccc015ad618f1, role_id=812d3c567c52469cb2b35709fbdc857c)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:01:45.799052 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b661657d-4562-41c4-a808-f29c1809bccb admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:01:45.818868 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 938/1902] 10.222.0.46 () {60 vars in 1152 bytes} [Thu Jun 23 16:01:45 2022] POST /identity/v3/projects => generated 373 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 16:01:45.823801 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa251df3-3889-46ef-81ce-de5261fcc144 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:01:45.841109 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 965/1903] 10.222.0.46 () {60 vars in 1482 bytes} [Thu Jun 23 16:01:45 2022] PUT /identity/v3/projects/f444fc15e9b14ff389c7ca73afabf3e1/users/59199bbe764d4f0bafaccc015ad618f1/roles/812d3c567c52469cb2b35709fbdc857c => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 16:01:45.864766 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abf0049-bd4d-41de-83a8-cf08b09d43a9 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:45.865257 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abf0049-bd4d-41de-83a8-cf08b09d43a9 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:45.865775 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abf0049-bd4d-41de-83a8-cf08b09d43a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:45.927539 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:01:45.960534 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-2abf0049-bd4d-41de-83a8-cf08b09d43a9 None None] MFA Rules not processed for user `59199bbe764d4f0bafaccc015ad618f1`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:01:45.969865 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-2abf0049-bd4d-41de-83a8-cf08b09d43a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:01:46.046470 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa251df3-3889-46ef-81ce-de5261fcc144 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:01:46.200641 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2138d13e-ae85-4060-a3ff-21ab3bd8b945 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:01:46.201649 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2138d13e-ae85-4060-a3ff-21ab3bd8b945 admin admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:46.201941 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2138d13e-ae85-4060-a3ff-21ab3bd8b945 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:46.202220 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2138d13e-ae85-4060-a3ff-21ab3bd8b945 admin admin] PATH_INFO: `/v3/users` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:46.203001 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2138d13e-ae85-4060-a3ff-21ab3bd8b945 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:01:46.205012 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2138d13e-ae85-4060-a3ff-21ab3bd8b945 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:01:46.258897 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 939/1904] 10.222.0.46 () {60 vars in 1143 bytes} [Thu Jun 23 16:01:45 2022] POST /identity/v3/users => generated 293 bytes in 439 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 16:01:46.264785 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2138d13e-ae85-4060-a3ff-21ab3bd8b945 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:01:46.282418 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2138d13e-ae85-4060-a3ff-21ab3bd8b945 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:01:46.289300 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1a9d46-45e6-4f85-9ad4-6c1c8925e637 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:01:46.290925 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1a9d46-45e6-4f85-9ad4-6c1c8925e637 admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:46.291535 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1a9d46-45e6-4f85-9ad4-6c1c8925e637 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:46.292030 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1a9d46-45e6-4f85-9ad4-6c1c8925e637 admin admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:46.293040 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1a9d46-45e6-4f85-9ad4-6c1c8925e637 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:01:46.294972 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1a9d46-45e6-4f85-9ad4-6c1c8925e637 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:01:46.312605 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 940/1905] 10.222.0.46 () {58 vars in 1121 bytes} [Thu Jun 23 16:01:46 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 16:01:46.320999 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1a9d46-45e6-4f85-9ad4-6c1c8925e637 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:01:46.338622 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1a9d46-45e6-4f85-9ad4-6c1c8925e637 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:01:46.344039 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 966/1906] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:01:45 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 482 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 16:01:46.365605 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abf0049-bd4d-41de-83a8-cf08b09d43a9 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:01:46.370050 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-2abf0049-bd4d-41de-83a8-cf08b09d43a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:01:46.380259 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46d222a-2bb9-48f4-a724-54822dea707a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:01:46.391906 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46d222a-2bb9-48f4-a724-54822dea707a admin admin] REQUEST_METHOD: `PUT` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:46.392493 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46d222a-2bb9-48f4-a724-54822dea707a admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:46.394293 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46d222a-2bb9-48f4-a724-54822dea707a admin admin] PATH_INFO: `/v3/projects/426f714fe02c4e4a9b95cb39181a988c/users/aa7f0bec8d5846919fe7cfdfd378f42b/roles/812d3c567c52469cb2b35709fbdc857c` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:46.443983 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46d222a-2bb9-48f4-a724-54822dea707a admin admin] RBAC: Authorizing `identity:create_grant(project_id=426f714fe02c4e4a9b95cb39181a988c, user_id=aa7f0bec8d5846919fe7cfdfd378f42b, role_id=812d3c567c52469cb2b35709fbdc857c)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:01:46.447020 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46d222a-2bb9-48f4-a724-54822dea707a admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:01:46.477864 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 941/1907] 10.222.0.46 () {60 vars in 1482 bytes} [Thu Jun 23 16:01:46 2022] PUT /identity/v3/projects/426f714fe02c4e4a9b95cb39181a988c/users/aa7f0bec8d5846919fe7cfdfd378f42b/roles/812d3c567c52469cb2b35709fbdc857c => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 16:01:46.481811 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46d222a-2bb9-48f4-a724-54822dea707a admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:01:46.540660 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:01:46.574853 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abf0049-bd4d-41de-83a8-cf08b09d43a9 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:01:46.611168 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46d222a-2bb9-48f4-a724-54822dea707a admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:01:46.619279 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9971b86e-98e9-4b94-adfa-e2616a1a0a53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:01:46.620419 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9971b86e-98e9-4b94-adfa-e2616a1a0a53 admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:46.620960 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9971b86e-98e9-4b94-adfa-e2616a1a0a53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:46.621426 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9971b86e-98e9-4b94-adfa-e2616a1a0a53 admin admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:46.622339 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9971b86e-98e9-4b94-adfa-e2616a1a0a53 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:01:46.624797 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9971b86e-98e9-4b94-adfa-e2616a1a0a53 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:01:46.641956 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 942/1908] 10.222.0.46 () {58 vars in 1121 bytes} [Thu Jun 23 16:01:46 2022] GET /identity/v3/roles => generated 1420 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 16:01:46.646772 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9971b86e-98e9-4b94-adfa-e2616a1a0a53 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:01:46.689081 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9971b86e-98e9-4b94-adfa-e2616a1a0a53 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:01:46.699244 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb4e0df-c63c-44f5-b7a7-65f565270aea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:01:46.702994 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb4e0df-c63c-44f5-b7a7-65f565270aea admin admin] REQUEST_METHOD: `PUT` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:46.702994 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb4e0df-c63c-44f5-b7a7-65f565270aea admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:46.703847 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb4e0df-c63c-44f5-b7a7-65f565270aea admin admin] PATH_INFO: `/v3/projects/426f714fe02c4e4a9b95cb39181a988c/users/aa7f0bec8d5846919fe7cfdfd378f42b/roles/812d3c567c52469cb2b35709fbdc857c` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:46.709096 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb4e0df-c63c-44f5-b7a7-65f565270aea admin admin] RBAC: Authorizing `identity:create_grant(project_id=426f714fe02c4e4a9b95cb39181a988c, user_id=aa7f0bec8d5846919fe7cfdfd378f42b, role_id=812d3c567c52469cb2b35709fbdc857c)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:01:46.715308 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e687984-870d-4b3c-afdb-987c4033865d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce18eaaba3a6477684fff1fd98ec8b51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:01:46.716308 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb4e0df-c63c-44f5-b7a7-65f565270aea admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:01:46.716467 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e687984-870d-4b3c-afdb-987c4033865d service cinder] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:46.722367 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e687984-870d-4b3c-afdb-987c4033865d service cinder] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:46.722684 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e687984-870d-4b3c-afdb-987c4033865d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:46.745847 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 943/1909] 10.222.0.46 () {60 vars in 1482 bytes} [Thu Jun 23 16:01:46 2022] PUT /identity/v3/projects/426f714fe02c4e4a9b95cb39181a988c/users/aa7f0bec8d5846919fe7cfdfd378f42b/roles/812d3c567c52469cb2b35709fbdc857c => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 16:01:46.757203 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d63a93d-f797-44f3-af43-19b63a81fd7d None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:46.758072 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d63a93d-f797-44f3-af43-19b63a81fd7d None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:46.758814 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d63a93d-f797-44f3-af43-19b63a81fd7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:46.861074 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:01:46.905483 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-8d63a93d-f797-44f3-af43-19b63a81fd7d None None] MFA Rules not processed for user `aa7f0bec8d5846919fe7cfdfd378f42b`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:01:46.913945 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-8d63a93d-f797-44f3-af43-19b63a81fd7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:01:47.088708 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e687984-870d-4b3c-afdb-987c4033865d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:01:47.098300 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e687984-870d-4b3c-afdb-987c4033865d service cinder] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:01:47.159166 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 967/1910] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jun 23 16:01:46 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 796 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 16:01:47.247358 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 944/1911] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:01:46 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 492 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 16:01:47.263893 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e687984-870d-4b3c-afdb-987c4033865d service cinder] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:01:47.287717 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e687984-870d-4b3c-afdb-987c4033865d service cinder] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:01:47.381136 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d63a93d-f797-44f3-af43-19b63a81fd7d None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:01:47.384752 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-8d63a93d-f797-44f3-af43-19b63a81fd7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:01:47.389587 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9360e89-a3b7-4410-bb80-e54cea2a7a96 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce18eaaba3a6477684fff1fd98ec8b51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:01:47.390450 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9360e89-a3b7-4410-bb80-e54cea2a7a96 service cinder] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:47.390623 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9360e89-a3b7-4410-bb80-e54cea2a7a96 service cinder] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:47.390920 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9360e89-a3b7-4410-bb80-e54cea2a7a96 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:47.421758 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d63a93d-f797-44f3-af43-19b63a81fd7d None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:01:47.433975 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcaef77-014d-4010-a2e1-9ef03be41700 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce18eaaba3a6477684fff1fd98ec8b51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:01:47.434750 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcaef77-014d-4010-a2e1-9ef03be41700 service cinder] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:47.435046 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcaef77-014d-4010-a2e1-9ef03be41700 service cinder] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:47.435327 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcaef77-014d-4010-a2e1-9ef03be41700 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:47.467799 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9360e89-a3b7-4410-bb80-e54cea2a7a96 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:01:47.468677 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9360e89-a3b7-4410-bb80-e54cea2a7a96 service cinder] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:01:47.493214 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 968/1912] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jun 23 16:01:47 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 16:01:47.561484 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dcaef77-014d-4010-a2e1-9ef03be41700 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:01:47.565305 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dcaef77-014d-4010-a2e1-9ef03be41700 service cinder] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:01:47.598051 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 945/1913] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jun 23 16:01:47 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 16:01:50.991724 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9360e89-a3b7-4410-bb80-e54cea2a7a96 service cinder] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:01:51.018381 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:01:51.095076 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcaef77-014d-4010-a2e1-9ef03be41700 service cinder] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:01:51.175553 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9360e89-a3b7-4410-bb80-e54cea2a7a96 service cinder] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:01:51.184928 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b3f43b-b1e9-46f4-998f-82e587719ae3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:01:51.186018 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b3f43b-b1e9-46f4-998f-82e587719ae3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:51.186385 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b3f43b-b1e9-46f4-998f-82e587719ae3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:51.186831 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b3f43b-b1e9-46f4-998f-82e587719ae3 admin admin] PATH_INFO: `/v3/users/8eef5f1c93bb44efa0af8597bb28d648` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:51.189348 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b3f43b-b1e9-46f4-998f-82e587719ae3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8eef5f1c93bb44efa0af8597bb28d648)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:01:51.193564 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b3f43b-b1e9-46f4-998f-82e587719ae3 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:01:51.241722 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcaef77-014d-4010-a2e1-9ef03be41700 service cinder] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:01:51.249654 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c40123-153d-4d31-8fb0-14e77433438c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:01:51.251055 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c40123-153d-4d31-8fb0-14e77433438c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:51.251564 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c40123-153d-4d31-8fb0-14e77433438c admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:51.252026 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c40123-153d-4d31-8fb0-14e77433438c admin admin] PATH_INFO: `/v3/users/ea2f6774a63e45e9acc1f5eb46b0850d` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:51.254507 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c40123-153d-4d31-8fb0-14e77433438c admin admin] RBAC: Authorizing `identity:delete_user(user_id=ea2f6774a63e45e9acc1f5eb46b0850d)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:01:51.256940 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c40123-153d-4d31-8fb0-14e77433438c admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:01:51.307945 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-95b3f43b-b1e9-46f4-998f-82e587719ae3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8eef5f1c93bb44efa0af8597bb28d648'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:01:51.326330 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:01:51.327117 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-95b3f43b-b1e9-46f4-998f-82e587719ae3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8eef5f1c93bb44efa0af8597bb28d648'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:01:51.328388 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-95b3f43b-b1e9-46f4-998f-82e587719ae3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8eef5f1c93bb44efa0af8597bb28d648'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:01:51.329568 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-95b3f43b-b1e9-46f4-998f-82e587719ae3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8eef5f1c93bb44efa0af8597bb28d648'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:01:51.352750 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-c9c40123-153d-4d31-8fb0-14e77433438c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ea2f6774a63e45e9acc1f5eb46b0850d'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:01:51.354029 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-c9c40123-153d-4d31-8fb0-14e77433438c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ea2f6774a63e45e9acc1f5eb46b0850d'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:01:51.357474 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-95b3f43b-b1e9-46f4-998f-82e587719ae3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8eef5f1c93bb44efa0af8597bb28d648'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:01:51.378565 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-c9c40123-153d-4d31-8fb0-14e77433438c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ea2f6774a63e45e9acc1f5eb46b0850d'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:01:51.383185 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 969/1914] 10.222.0.46 () {58 vars in 1223 bytes} [Thu Jun 23 16:01:50 2022] DELETE /identity/v3/users/8eef5f1c93bb44efa0af8597bb28d648 => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 16:01:51.390683 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ca4854-9678-4b0e-b09e-bd8d3ca7c37f None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:51.391225 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ca4854-9678-4b0e-b09e-bd8d3ca7c37f None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:51.391741 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ca4854-9678-4b0e-b09e-bd8d3ca7c37f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:51.396592 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-c9c40123-153d-4d31-8fb0-14e77433438c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ea2f6774a63e45e9acc1f5eb46b0850d'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:01:51.398001 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-c9c40123-153d-4d31-8fb0-14e77433438c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ea2f6774a63e45e9acc1f5eb46b0850d'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:01:51.426177 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-01ca4854-9678-4b0e-b09e-bd8d3ca7c37f None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:01:51.433393 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-01ca4854-9678-4b0e-b09e-bd8d3ca7c37f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:01:51.438930 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 946/1915] 10.222.0.46 () {58 vars in 1223 bytes} [Thu Jun 23 16:01:51 2022] DELETE /identity/v3/users/ea2f6774a63e45e9acc1f5eb46b0850d => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 16:01:51.440910 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00da8889-2861-4a3a-8cb8-0a2dd6f08a7e None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:51.441217 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00da8889-2861-4a3a-8cb8-0a2dd6f08a7e None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:51.441486 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00da8889-2861-4a3a-8cb8-0a2dd6f08a7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:51.465308 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-00da8889-2861-4a3a-8cb8-0a2dd6f08a7e None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:01:51.469215 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-00da8889-2861-4a3a-8cb8-0a2dd6f08a7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:01:51.565127 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 970/1916] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:01:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 16:01:51.581335 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ca4854-9678-4b0e-b09e-bd8d3ca7c37f None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:01:51.582931 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-01ca4854-9678-4b0e-b09e-bd8d3ca7c37f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:01:51.599163 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 947/1917] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:01:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 16:01:51.611004 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00da8889-2861-4a3a-8cb8-0a2dd6f08a7e None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:01:51.612918 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-00da8889-2861-4a3a-8cb8-0a2dd6f08a7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:01:51.659627 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:01:51.720329 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00da8889-2861-4a3a-8cb8-0a2dd6f08a7e None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:01:51.721713 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ca4854-9678-4b0e-b09e-bd8d3ca7c37f None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:01:51.729157 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498b2ed1-b8e9-4f1b-9385-16ecaaf2d5a3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51784064951a4619ae09b9167fb0fd0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:01:51.730261 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498b2ed1-b8e9-4f1b-9385-16ecaaf2d5a3 service neutron] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:51.730782 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498b2ed1-b8e9-4f1b-9385-16ecaaf2d5a3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:51.731085 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381dc975-d72d-4666-9f7d-6533ec19c696 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51784064951a4619ae09b9167fb0fd0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:01:51.731267 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498b2ed1-b8e9-4f1b-9385-16ecaaf2d5a3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:51.732532 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381dc975-d72d-4666-9f7d-6533ec19c696 service neutron] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:51.734001 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381dc975-d72d-4666-9f7d-6533ec19c696 service neutron] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:51.734001 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381dc975-d72d-4666-9f7d-6533ec19c696 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:51.741897 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498b2ed1-b8e9-4f1b-9385-16ecaaf2d5a3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:01:51.743659 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498b2ed1-b8e9-4f1b-9385-16ecaaf2d5a3 service neutron] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:01:51.754578 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381dc975-d72d-4666-9f7d-6533ec19c696 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:01:51.755686 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 948/1918] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jun 23 16:01:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 16:01:51.759403 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381dc975-d72d-4666-9f7d-6533ec19c696 service neutron] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:01:51.782896 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 971/1919] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jun 23 16:01:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 16:01:52.279773 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498b2ed1-b8e9-4f1b-9385-16ecaaf2d5a3 service neutron] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:01:52.285806 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-498b2ed1-b8e9-4f1b-9385-16ecaaf2d5a3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:01:52.340547 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498b2ed1-b8e9-4f1b-9385-16ecaaf2d5a3 service neutron] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:01:52.342325 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381dc975-d72d-4666-9f7d-6533ec19c696 service neutron] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:01:52.344420 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-381dc975-d72d-4666-9f7d-6533ec19c696 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:01:52.354241 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78106d4c-d205-4f0d-8031-fd056269ab48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:01:52.356166 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78106d4c-d205-4f0d-8031-fd056269ab48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:52.357022 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78106d4c-d205-4f0d-8031-fd056269ab48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:52.357514 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78106d4c-d205-4f0d-8031-fd056269ab48 admin admin] PATH_INFO: `/v3/projects/ed4025121c444d1f86fdcb5d4e7b03e2` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:52.363102 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78106d4c-d205-4f0d-8031-fd056269ab48 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ed4025121c444d1f86fdcb5d4e7b03e2)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:01:52.368691 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381dc975-d72d-4666-9f7d-6533ec19c696 service neutron] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:01:52.370110 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78106d4c-d205-4f0d-8031-fd056269ab48 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:01:52.375289 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12da28d-9e91-48bb-bb1b-a25acf8f6dd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:01:52.376634 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12da28d-9e91-48bb-bb1b-a25acf8f6dd2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:52.376634 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12da28d-9e91-48bb-bb1b-a25acf8f6dd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:52.376931 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12da28d-9e91-48bb-bb1b-a25acf8f6dd2 admin admin] PATH_INFO: `/v3/projects/0a416742729e4eaa95846ecf5a61c677` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:52.378294 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12da28d-9e91-48bb-bb1b-a25acf8f6dd2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0a416742729e4eaa95846ecf5a61c677)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:01:52.380853 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12da28d-9e91-48bb-bb1b-a25acf8f6dd2 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:01:52.451201 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-78106d4c-d205-4f0d-8031-fd056269ab48 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ed4025121c444d1f86fdcb5d4e7b03e2', 'request_id': 'req-78106d4c-d205-4f0d-8031-fd056269ab48'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:01:52.455640 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:01:52.461001 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-78106d4c-d205-4f0d-8031-fd056269ab48 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ed4025121c444d1f86fdcb5d4e7b03e2', 'request_id': 'req-78106d4c-d205-4f0d-8031-fd056269ab48'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:01:52.464374 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-e12da28d-9e91-48bb-bb1b-a25acf8f6dd2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0a416742729e4eaa95846ecf5a61c677', 'request_id': 'req-e12da28d-9e91-48bb-bb1b-a25acf8f6dd2'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:01:52.471802 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-e12da28d-9e91-48bb-bb1b-a25acf8f6dd2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0a416742729e4eaa95846ecf5a61c677', 'request_id': 'req-e12da28d-9e91-48bb-bb1b-a25acf8f6dd2'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:01:52.475797 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-78106d4c-d205-4f0d-8031-fd056269ab48 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ed4025121c444d1f86fdcb5d4e7b03e2', 'request_id': 'req-78106d4c-d205-4f0d-8031-fd056269ab48'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:01:52.483735 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-e12da28d-9e91-48bb-bb1b-a25acf8f6dd2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0a416742729e4eaa95846ecf5a61c677', 'request_id': 'req-e12da28d-9e91-48bb-bb1b-a25acf8f6dd2'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:01:52.492113 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-78106d4c-d205-4f0d-8031-fd056269ab48 admin admin] The token cache is being invalidate because project ed4025121c444d1f86fdcb5d4e7b03e2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80154) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 16:01:52.492412 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-78106d4c-d205-4f0d-8031-fd056269ab48 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:01:52.493754 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-78106d4c-d205-4f0d-8031-fd056269ab48 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:01:52.495972 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 949/1920] 10.222.0.46 () {58 vars in 1232 bytes} [Thu Jun 23 16:01:52 2022] DELETE /identity/v3/projects/ed4025121c444d1f86fdcb5d4e7b03e2 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 16:01:52.500376 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78106d4c-d205-4f0d-8031-fd056269ab48 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:01:52.500686 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-e12da28d-9e91-48bb-bb1b-a25acf8f6dd2 admin admin] The token cache is being invalidate because project 0a416742729e4eaa95846ecf5a61c677 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80153) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 16:01:52.500987 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-e12da28d-9e91-48bb-bb1b-a25acf8f6dd2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:01:52.501660 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-e12da28d-9e91-48bb-bb1b-a25acf8f6dd2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:01:52.502629 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-78106d4c-d205-4f0d-8031-fd056269ab48 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:01:52.503608 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 972/1921] 10.222.0.46 () {58 vars in 1232 bytes} [Thu Jun 23 16:01:52 2022] DELETE /identity/v3/projects/0a416742729e4eaa95846ecf5a61c677 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 16:01:52.508303 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12da28d-9e91-48bb-bb1b-a25acf8f6dd2 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:01:52.510644 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-e12da28d-9e91-48bb-bb1b-a25acf8f6dd2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:01:52.624729 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78106d4c-d205-4f0d-8031-fd056269ab48 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:01:52.631030 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1430e3-38c8-406c-b1d9-ef50c092f7e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:01:52.632248 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1430e3-38c8-406c-b1d9-ef50c092f7e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:52.632649 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1430e3-38c8-406c-b1d9-ef50c092f7e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:52.633274 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1430e3-38c8-406c-b1d9-ef50c092f7e0 admin admin] PATH_INFO: `/v3/users/aa7f0bec8d5846919fe7cfdfd378f42b` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:52.635200 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1430e3-38c8-406c-b1d9-ef50c092f7e0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aa7f0bec8d5846919fe7cfdfd378f42b)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:01:52.637099 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1430e3-38c8-406c-b1d9-ef50c092f7e0 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:01:52.644312 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12da28d-9e91-48bb-bb1b-a25acf8f6dd2 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:01:52.656258 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40086ed5-c59d-41a9-a886-2db807c27e29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:01:52.657170 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40086ed5-c59d-41a9-a886-2db807c27e29 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:52.657454 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40086ed5-c59d-41a9-a886-2db807c27e29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:52.657730 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40086ed5-c59d-41a9-a886-2db807c27e29 admin admin] PATH_INFO: `/v3/users/59199bbe764d4f0bafaccc015ad618f1` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:52.659063 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40086ed5-c59d-41a9-a886-2db807c27e29 admin admin] RBAC: Authorizing `identity:delete_user(user_id=59199bbe764d4f0bafaccc015ad618f1)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:01:52.661050 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40086ed5-c59d-41a9-a886-2db807c27e29 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:01:52.707368 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-4f1430e3-38c8-406c-b1d9-ef50c092f7e0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aa7f0bec8d5846919fe7cfdfd378f42b'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:01:52.708281 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-4f1430e3-38c8-406c-b1d9-ef50c092f7e0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aa7f0bec8d5846919fe7cfdfd378f42b'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:01:52.716888 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-4f1430e3-38c8-406c-b1d9-ef50c092f7e0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aa7f0bec8d5846919fe7cfdfd378f42b'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:01:52.726283 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-4f1430e3-38c8-406c-b1d9-ef50c092f7e0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aa7f0bec8d5846919fe7cfdfd378f42b'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:01:52.727622 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-4f1430e3-38c8-406c-b1d9-ef50c092f7e0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aa7f0bec8d5846919fe7cfdfd378f42b'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:01:52.730330 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-40086ed5-c59d-41a9-a886-2db807c27e29 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '59199bbe764d4f0bafaccc015ad618f1'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:01:52.740988 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:01:52.746351 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-40086ed5-c59d-41a9-a886-2db807c27e29 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '59199bbe764d4f0bafaccc015ad618f1'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:01:52.748621 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-40086ed5-c59d-41a9-a886-2db807c27e29 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '59199bbe764d4f0bafaccc015ad618f1'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:01:52.748804 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 950/1922] 10.222.0.46 () {58 vars in 1223 bytes} [Thu Jun 23 16:01:52 2022] DELETE /identity/v3/users/aa7f0bec8d5846919fe7cfdfd378f42b => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 16:01:52.749887 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-40086ed5-c59d-41a9-a886-2db807c27e29 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '59199bbe764d4f0bafaccc015ad618f1'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:01:52.774604 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-40086ed5-c59d-41a9-a886-2db807c27e29 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '59199bbe764d4f0bafaccc015ad618f1'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:01:52.796235 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 973/1923] 10.222.0.46 () {58 vars in 1223 bytes} [Thu Jun 23 16:01:52 2022] DELETE /identity/v3/users/59199bbe764d4f0bafaccc015ad618f1 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 16:01:53.178309 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1430e3-38c8-406c-b1d9-ef50c092f7e0 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:01:53.180937 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-4f1430e3-38c8-406c-b1d9-ef50c092f7e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:01:53.243874 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40086ed5-c59d-41a9-a886-2db807c27e29 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:01:53.246474 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-40086ed5-c59d-41a9-a886-2db807c27e29 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:01:53.326141 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:01:53.333121 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1430e3-38c8-406c-b1d9-ef50c092f7e0 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:01:53.377282 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40086ed5-c59d-41a9-a886-2db807c27e29 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:01:53.386810 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fab3113-dc89-45a0-b3b8-74504f9e7b39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:01:53.388280 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fab3113-dc89-45a0-b3b8-74504f9e7b39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:53.388795 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fab3113-dc89-45a0-b3b8-74504f9e7b39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:53.389799 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fab3113-dc89-45a0-b3b8-74504f9e7b39 admin admin] PATH_INFO: `/v3/projects/f444fc15e9b14ff389c7ca73afabf3e1` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:53.391295 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fab3113-dc89-45a0-b3b8-74504f9e7b39 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f444fc15e9b14ff389c7ca73afabf3e1)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:01:53.395172 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fab3113-dc89-45a0-b3b8-74504f9e7b39 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:01:53.413678 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d53dd61-c04d-4b26-97a5-298f8ae07595 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:01:53.414430 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d53dd61-c04d-4b26-97a5-298f8ae07595 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:53.414755 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d53dd61-c04d-4b26-97a5-298f8ae07595 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:53.415350 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d53dd61-c04d-4b26-97a5-298f8ae07595 admin admin] PATH_INFO: `/v3/projects/426f714fe02c4e4a9b95cb39181a988c` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:53.416835 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d53dd61-c04d-4b26-97a5-298f8ae07595 admin admin] RBAC: Authorizing `identity:delete_project(project_id=426f714fe02c4e4a9b95cb39181a988c)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:01:53.418855 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d53dd61-c04d-4b26-97a5-298f8ae07595 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:01:53.480951 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-0fab3113-dc89-45a0-b3b8-74504f9e7b39 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f444fc15e9b14ff389c7ca73afabf3e1', 'request_id': 'req-0fab3113-dc89-45a0-b3b8-74504f9e7b39'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:01:53.489637 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-0fab3113-dc89-45a0-b3b8-74504f9e7b39 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f444fc15e9b14ff389c7ca73afabf3e1', 'request_id': 'req-0fab3113-dc89-45a0-b3b8-74504f9e7b39'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:01:53.490338 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-5d53dd61-c04d-4b26-97a5-298f8ae07595 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '426f714fe02c4e4a9b95cb39181a988c', 'request_id': 'req-5d53dd61-c04d-4b26-97a5-298f8ae07595'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:01:53.498026 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-5d53dd61-c04d-4b26-97a5-298f8ae07595 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '426f714fe02c4e4a9b95cb39181a988c', 'request_id': 'req-5d53dd61-c04d-4b26-97a5-298f8ae07595'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:01:53.506144 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-0fab3113-dc89-45a0-b3b8-74504f9e7b39 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f444fc15e9b14ff389c7ca73afabf3e1', 'request_id': 'req-0fab3113-dc89-45a0-b3b8-74504f9e7b39'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:01:53.509226 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-5d53dd61-c04d-4b26-97a5-298f8ae07595 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '426f714fe02c4e4a9b95cb39181a988c', 'request_id': 'req-5d53dd61-c04d-4b26-97a5-298f8ae07595'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:01:53.531468 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-0fab3113-dc89-45a0-b3b8-74504f9e7b39 admin admin] The token cache is being invalidate because project f444fc15e9b14ff389c7ca73afabf3e1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80153) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 16:01:53.531843 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-0fab3113-dc89-45a0-b3b8-74504f9e7b39 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:01:53.532825 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-0fab3113-dc89-45a0-b3b8-74504f9e7b39 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:01:53.535092 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-5d53dd61-c04d-4b26-97a5-298f8ae07595 admin admin] The token cache is being invalidate because project 426f714fe02c4e4a9b95cb39181a988c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80154) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 16:01:53.535679 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-5d53dd61-c04d-4b26-97a5-298f8ae07595 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:01:53.536211 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 974/1924] 10.222.0.46 () {58 vars in 1232 bytes} [Thu Jun 23 16:01:53 2022] DELETE /identity/v3/projects/f444fc15e9b14ff389c7ca73afabf3e1 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 16:01:53.536943 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-5d53dd61-c04d-4b26-97a5-298f8ae07595 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:01:53.540008 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 951/1925] 10.222.0.46 () {58 vars in 1232 bytes} [Thu Jun 23 16:01:53 2022] DELETE /identity/v3/projects/426f714fe02c4e4a9b95cb39181a988c => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 16:01:53.542578 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9af9e6-7ca1-4e3e-a87e-a6da2a2ac401 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:53.543196 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9af9e6-7ca1-4e3e-a87e-a6da2a2ac401 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:53.543685 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9af9e6-7ca1-4e3e-a87e-a6da2a2ac401 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:53.547843 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2beaf48-8596-416b-925b-1156c17a2275 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:53.548378 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2beaf48-8596-416b-925b-1156c17a2275 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:53.548884 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2beaf48-8596-416b-925b-1156c17a2275 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:53.575013 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-ec9af9e6-7ca1-4e3e-a87e-a6da2a2ac401 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:01:53.577539 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-ec9af9e6-7ca1-4e3e-a87e-a6da2a2ac401 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:01:53.577666 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-a2beaf48-8596-416b-925b-1156c17a2275 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:01:53.578939 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-a2beaf48-8596-416b-925b-1156c17a2275 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:01:53.674114 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 952/1926] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:01:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 16:01:53.687686 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2165a3fb-04bc-4632-8569-805eb2195872 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:53.687980 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2165a3fb-04bc-4632-8569-805eb2195872 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:53.688240 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2165a3fb-04bc-4632-8569-805eb2195872 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:53.705890 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 975/1927] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:01:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 16:01:53.707740 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9e6141-22ec-4511-a7dc-d76ebf12458a None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:53.708032 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9e6141-22ec-4511-a7dc-d76ebf12458a None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:53.708316 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9e6141-22ec-4511-a7dc-d76ebf12458a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:53.715563 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:01:53.731671 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-2165a3fb-04bc-4632-8569-805eb2195872 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:01:53.735438 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-2165a3fb-04bc-4632-8569-805eb2195872 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:01:53.739092 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-9a9e6141-22ec-4511-a7dc-d76ebf12458a None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:01:53.742182 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-9a9e6141-22ec-4511-a7dc-d76ebf12458a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:01:53.747210 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 953/1928] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:01:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 16:01:53.755974 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2165a3fb-04bc-4632-8569-805eb2195872 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:01:53.757994 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 976/1929] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:01:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 16:01:53.764547 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9e6141-22ec-4511-a7dc-d76ebf12458a None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:01:53.771359 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2165a3fb-04bc-4632-8569-805eb2195872 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:01:53.778102 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b0296e-d15a-4a71-a823-165b38639d6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:01:53.779108 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b0296e-d15a-4a71-a823-165b38639d6b admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:53.779595 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b0296e-d15a-4a71-a823-165b38639d6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:53.780068 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b0296e-d15a-4a71-a823-165b38639d6b admin admin] PATH_INFO: `/v3/domains` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:53.781248 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b0296e-d15a-4a71-a823-165b38639d6b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 16:01:53.781994 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b0296e-d15a-4a71-a823-165b38639d6b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:01:53.782595 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9e6141-22ec-4511-a7dc-d76ebf12458a None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:01:53.784141 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b0296e-d15a-4a71-a823-165b38639d6b admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:01:53.791305 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348ac3a7-9bd4-410f-aeb1-e148e0e12bb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:01:53.792316 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348ac3a7-9bd4-410f-aeb1-e148e0e12bb1 admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:53.792596 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348ac3a7-9bd4-410f-aeb1-e148e0e12bb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:53.792886 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348ac3a7-9bd4-410f-aeb1-e148e0e12bb1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:53.794298 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348ac3a7-9bd4-410f-aeb1-e148e0e12bb1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 16:01:53.794879 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348ac3a7-9bd4-410f-aeb1-e148e0e12bb1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:01:53.797791 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348ac3a7-9bd4-410f-aeb1-e148e0e12bb1 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:01:53.798571 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 954/1930] 10.222.0.46 () {58 vars in 1152 bytes} [Thu Jun 23 16:01:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 16:01:53.802965 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b0296e-d15a-4a71-a823-165b38639d6b admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:01:53.814020 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b0296e-d15a-4a71-a823-165b38639d6b admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:01:53.819257 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98987e5-e141-4d3b-be6e-a690b7e9e88a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:01:53.819897 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98987e5-e141-4d3b-be6e-a690b7e9e88a admin admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:53.820181 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98987e5-e141-4d3b-be6e-a690b7e9e88a admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:53.824278 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98987e5-e141-4d3b-be6e-a690b7e9e88a admin admin] PATH_INFO: `/v3/projects` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:53.824534 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 977/1931] 10.222.0.46 () {58 vars in 1152 bytes} [Thu Jun 23 16:01:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 16:01:53.824633 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98987e5-e141-4d3b-be6e-a690b7e9e88a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:01:53.824633 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98987e5-e141-4d3b-be6e-a690b7e9e88a admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:01:53.825934 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348ac3a7-9bd4-410f-aeb1-e148e0e12bb1 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:01:53.838747 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:01:53.843070 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348ac3a7-9bd4-410f-aeb1-e148e0e12bb1 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:01:53.851632 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4538ffa-4171-40b6-b749-32e34dd91af8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:01:53.852753 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4538ffa-4171-40b6-b749-32e34dd91af8 admin admin] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:53.853194 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4538ffa-4171-40b6-b749-32e34dd91af8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:53.853659 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4538ffa-4171-40b6-b749-32e34dd91af8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:53.854838 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4538ffa-4171-40b6-b749-32e34dd91af8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:01:53.860026 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 955/1932] 10.222.0.46 () {60 vars in 1152 bytes} [Thu Jun 23 16:01:53 2022] POST /identity/v3/projects => generated 375 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 16:01:53.860219 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4538ffa-4171-40b6-b749-32e34dd91af8 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:01:53.861194 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98987e5-e141-4d3b-be6e-a690b7e9e88a admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:01:53.899786 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 978/1933] 10.222.0.46 () {60 vars in 1152 bytes} [Thu Jun 23 16:01:53 2022] POST /identity/v3/projects => generated 377 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 16:01:53.905624 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4538ffa-4171-40b6-b749-32e34dd91af8 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:01:53.946368 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98987e5-e141-4d3b-be6e-a690b7e9e88a admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:01:54.004297 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4538ffa-4171-40b6-b749-32e34dd91af8 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:01:54.010000 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f0a36c-ddf7-4c85-a3b2-7d83b7642ce6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:01:54.010665 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f0a36c-ddf7-4c85-a3b2-7d83b7642ce6 admin admin] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:54.010962 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f0a36c-ddf7-4c85-a3b2-7d83b7642ce6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:54.011248 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f0a36c-ddf7-4c85-a3b2-7d83b7642ce6 admin admin] PATH_INFO: `/v3/users` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:54.012002 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f0a36c-ddf7-4c85-a3b2-7d83b7642ce6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:01:54.012321 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32862372-d545-4e94-8d32-cd6e8ba7394a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:01:54.013036 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32862372-d545-4e94-8d32-cd6e8ba7394a admin admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:54.013319 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32862372-d545-4e94-8d32-cd6e8ba7394a admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:54.013599 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32862372-d545-4e94-8d32-cd6e8ba7394a admin admin] PATH_INFO: `/v3/users` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:54.014006 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f0a36c-ddf7-4c85-a3b2-7d83b7642ce6 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:01:54.014304 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32862372-d545-4e94-8d32-cd6e8ba7394a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:01:54.016393 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32862372-d545-4e94-8d32-cd6e8ba7394a admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:01:54.048540 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 979/1934] 10.222.0.46 () {60 vars in 1143 bytes} [Thu Jun 23 16:01:53 2022] POST /identity/v3/users => generated 289 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 16:01:54.048750 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 956/1935] 10.222.0.46 () {60 vars in 1143 bytes} [Thu Jun 23 16:01:53 2022] POST /identity/v3/users => generated 288 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 16:01:54.051470 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f0a36c-ddf7-4c85-a3b2-7d83b7642ce6 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:01:54.051698 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32862372-d545-4e94-8d32-cd6e8ba7394a admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:01:54.063003 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f0a36c-ddf7-4c85-a3b2-7d83b7642ce6 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:01:54.063275 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32862372-d545-4e94-8d32-cd6e8ba7394a admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:01:54.068646 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ea77ce-18b1-4edf-bcec-bb0fe71e5a62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:01:54.069212 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9a1e90-261a-4895-b400-c89a7a926851 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:01:54.069326 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ea77ce-18b1-4edf-bcec-bb0fe71e5a62 admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:54.069608 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ea77ce-18b1-4edf-bcec-bb0fe71e5a62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:54.069892 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ea77ce-18b1-4edf-bcec-bb0fe71e5a62 admin admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:54.070224 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9a1e90-261a-4895-b400-c89a7a926851 admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:54.070609 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9a1e90-261a-4895-b400-c89a7a926851 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:54.070762 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ea77ce-18b1-4edf-bcec-bb0fe71e5a62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:01:54.071010 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9a1e90-261a-4895-b400-c89a7a926851 admin admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:54.071815 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9a1e90-261a-4895-b400-c89a7a926851 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:01:54.072275 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ea77ce-18b1-4edf-bcec-bb0fe71e5a62 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:01:54.073891 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9a1e90-261a-4895-b400-c89a7a926851 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:01:54.086514 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 980/1936] 10.222.0.46 () {58 vars in 1121 bytes} [Thu Jun 23 16:01:54 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 16:01:54.086776 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 957/1937] 10.222.0.46 () {58 vars in 1121 bytes} [Thu Jun 23 16:01:54 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 16:01:54.087735 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ea77ce-18b1-4edf-bcec-bb0fe71e5a62 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:01:54.091734 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9a1e90-261a-4895-b400-c89a7a926851 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:01:54.106068 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ea77ce-18b1-4edf-bcec-bb0fe71e5a62 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:01:54.108184 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9a1e90-261a-4895-b400-c89a7a926851 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:01:54.112253 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bc1a23-f33f-4552-9383-9b2f96601e6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:01:54.113060 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bc1a23-f33f-4552-9383-9b2f96601e6c admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:54.113359 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bc1a23-f33f-4552-9383-9b2f96601e6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:54.113640 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bc1a23-f33f-4552-9383-9b2f96601e6c admin admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:54.114423 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bc1a23-f33f-4552-9383-9b2f96601e6c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:01:54.116430 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bc1a23-f33f-4552-9383-9b2f96601e6c admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:01:54.117519 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7cf52a-f1eb-4288-8e3c-9dfe2264e07a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:01:54.118842 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7cf52a-f1eb-4288-8e3c-9dfe2264e07a admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:54.119250 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7cf52a-f1eb-4288-8e3c-9dfe2264e07a admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:54.119663 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7cf52a-f1eb-4288-8e3c-9dfe2264e07a admin admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:54.120409 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7cf52a-f1eb-4288-8e3c-9dfe2264e07a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:01:54.122326 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7cf52a-f1eb-4288-8e3c-9dfe2264e07a admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:01:54.134436 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 981/1938] 10.222.0.46 () {58 vars in 1121 bytes} [Thu Jun 23 16:01:54 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 16:01:54.136816 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 958/1939] 10.222.0.46 () {58 vars in 1121 bytes} [Thu Jun 23 16:01:54 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 16:01:54.141147 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7cf52a-f1eb-4288-8e3c-9dfe2264e07a admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:01:54.141731 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bc1a23-f33f-4552-9383-9b2f96601e6c admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:01:54.155678 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7cf52a-f1eb-4288-8e3c-9dfe2264e07a admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:01:54.157919 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bc1a23-f33f-4552-9383-9b2f96601e6c admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:01:54.163816 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11e54ce-8b54-4d69-8830-a00c0041af12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:01:54.165174 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e551e3e-c3c2-4b47-9219-fed734a78966 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:01:54.165531 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11e54ce-8b54-4d69-8830-a00c0041af12 admin admin] REQUEST_METHOD: `PUT` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:54.165993 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11e54ce-8b54-4d69-8830-a00c0041af12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:54.166277 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e551e3e-c3c2-4b47-9219-fed734a78966 admin admin] REQUEST_METHOD: `PUT` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:54.166395 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11e54ce-8b54-4d69-8830-a00c0041af12 admin admin] PATH_INFO: `/v3/projects/5f09d9a69e08481cb299bdcfdf6efca4/users/661a7520a6de4399b19afecbad7c6164/roles/e2a5bbf3200a4d4091d72bc7ff78c34d` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:54.166673 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e551e3e-c3c2-4b47-9219-fed734a78966 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:54.167044 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e551e3e-c3c2-4b47-9219-fed734a78966 admin admin] PATH_INFO: `/v3/projects/bc64beda3ae44fa9ba126c1eabdd536f/users/1db1808db098471facffc352c3a7e52c/roles/e2a5bbf3200a4d4091d72bc7ff78c34d` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:54.186495 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11e54ce-8b54-4d69-8830-a00c0041af12 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f09d9a69e08481cb299bdcfdf6efca4, user_id=661a7520a6de4399b19afecbad7c6164, role_id=e2a5bbf3200a4d4091d72bc7ff78c34d)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:01:54.187586 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e551e3e-c3c2-4b47-9219-fed734a78966 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc64beda3ae44fa9ba126c1eabdd536f, user_id=1db1808db098471facffc352c3a7e52c, role_id=e2a5bbf3200a4d4091d72bc7ff78c34d)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:01:54.189233 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11e54ce-8b54-4d69-8830-a00c0041af12 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:01:54.190259 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e551e3e-c3c2-4b47-9219-fed734a78966 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:01:54.208033 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 959/1940] 10.222.0.46 () {60 vars in 1482 bytes} [Thu Jun 23 16:01:54 2022] PUT /identity/v3/projects/5f09d9a69e08481cb299bdcfdf6efca4/users/661a7520a6de4399b19afecbad7c6164/roles/e2a5bbf3200a4d4091d72bc7ff78c34d => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 16:01:54.214257 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 982/1941] 10.222.0.46 () {60 vars in 1482 bytes} [Thu Jun 23 16:01:54 2022] PUT /identity/v3/projects/bc64beda3ae44fa9ba126c1eabdd536f/users/1db1808db098471facffc352c3a7e52c/roles/e2a5bbf3200a4d4091d72bc7ff78c34d => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 16:01:54.220490 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bedde9-4569-4491-b9c1-2ba36ceda692 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:54.220778 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bedde9-4569-4491-b9c1-2ba36ceda692 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:54.221059 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bedde9-4569-4491-b9c1-2ba36ceda692 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:54.228287 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba11503-b88d-42b3-921e-7590c56e9ec6 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:54.228582 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba11503-b88d-42b3-921e-7590c56e9ec6 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:54.228913 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba11503-b88d-42b3-921e-7590c56e9ec6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:54.263448 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-c1bedde9-4569-4491-b9c1-2ba36ceda692 None None] MFA Rules not processed for user `661a7520a6de4399b19afecbad7c6164`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:01:54.263818 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-eba11503-b88d-42b3-921e-7590c56e9ec6 None None] MFA Rules not processed for user `1db1808db098471facffc352c3a7e52c`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:01:54.265051 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-eba11503-b88d-42b3-921e-7590c56e9ec6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:01:54.265680 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-c1bedde9-4569-4491-b9c1-2ba36ceda692 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:01:54.326667 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:01:54.355959 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:01:54.378892 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 983/1942] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:01:54 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 16:01:54.382105 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba11503-b88d-42b3-921e-7590c56e9ec6 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:01:54.386891 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 960/1943] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:01:54 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 16:01:54.406844 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bedde9-4569-4491-b9c1-2ba36ceda692 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:01:54.408366 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-c1bedde9-4569-4491-b9c1-2ba36ceda692 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:01:54.465577 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba11503-b88d-42b3-921e-7590c56e9ec6 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:01:54.471290 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47a9e3d-bbfa-4122-9491-69c4c7bec2c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:01:54.472143 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47a9e3d-bbfa-4122-9491-69c4c7bec2c3 admin admin] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:54.472519 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47a9e3d-bbfa-4122-9491-69c4c7bec2c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:54.472884 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47a9e3d-bbfa-4122-9491-69c4c7bec2c3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:54.473699 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47a9e3d-bbfa-4122-9491-69c4c7bec2c3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:01:54.476199 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47a9e3d-bbfa-4122-9491-69c4c7bec2c3 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:01:54.493975 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bedde9-4569-4491-b9c1-2ba36ceda692 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:01:54.499865 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c485a6c-6319-432d-8f37-5437703f68d6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce18eaaba3a6477684fff1fd98ec8b51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:01:54.500566 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c485a6c-6319-432d-8f37-5437703f68d6 service cinder] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:54.500857 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c485a6c-6319-432d-8f37-5437703f68d6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:54.501140 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c485a6c-6319-432d-8f37-5437703f68d6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:54.511255 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c485a6c-6319-432d-8f37-5437703f68d6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:01:54.514853 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 984/1944] 10.222.0.46 () {60 vars in 1152 bytes} [Thu Jun 23 16:01:54 2022] POST /identity/v3/projects => generated 377 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 16:01:54.515020 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c485a6c-6319-432d-8f37-5437703f68d6 service cinder] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:01:54.516662 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47a9e3d-bbfa-4122-9491-69c4c7bec2c3 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:01:54.525894 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 961/1945] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jun 23 16:01:54 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 16:01:54.733520 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47a9e3d-bbfa-4122-9491-69c4c7bec2c3 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:01:54.757637 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d856b1ed-9bd1-40d4-9736-29cd51d0c4b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:01:54.763640 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d856b1ed-9bd1-40d4-9736-29cd51d0c4b5 admin admin] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:54.764012 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d856b1ed-9bd1-40d4-9736-29cd51d0c4b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:54.764300 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d856b1ed-9bd1-40d4-9736-29cd51d0c4b5 admin admin] PATH_INFO: `/v3/users` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:54.767642 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d856b1ed-9bd1-40d4-9736-29cd51d0c4b5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:01:54.776585 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d856b1ed-9bd1-40d4-9736-29cd51d0c4b5 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:01:54.855042 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 985/1946] 10.222.0.46 () {60 vars in 1143 bytes} [Thu Jun 23 16:01:54 2022] POST /identity/v3/users => generated 295 bytes in 339 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 16:01:54.855338 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c485a6c-6319-432d-8f37-5437703f68d6 service cinder] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:01:54.880158 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c485a6c-6319-432d-8f37-5437703f68d6 service cinder] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:01:54.895758 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15745cb1-00fd-414b-babe-2ecfb4b468c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:01:54.897929 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15745cb1-00fd-414b-babe-2ecfb4b468c0 admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:54.899612 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15745cb1-00fd-414b-babe-2ecfb4b468c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:54.899919 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15745cb1-00fd-414b-babe-2ecfb4b468c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:54.900939 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15745cb1-00fd-414b-babe-2ecfb4b468c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:01:54.904814 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15745cb1-00fd-414b-babe-2ecfb4b468c0 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:01:54.928957 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d856b1ed-9bd1-40d4-9736-29cd51d0c4b5 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:01:54.934956 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 962/1947] 10.222.0.46 () {58 vars in 1121 bytes} [Thu Jun 23 16:01:54 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 16:01:54.948883 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d856b1ed-9bd1-40d4-9736-29cd51d0c4b5 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:01:54.972505 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c683ccca-1b1c-426d-8b06-e5413024f41b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:01:54.975132 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c683ccca-1b1c-426d-8b06-e5413024f41b admin admin] REQUEST_METHOD: `PUT` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:54.975426 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c683ccca-1b1c-426d-8b06-e5413024f41b admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:54.977410 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c683ccca-1b1c-426d-8b06-e5413024f41b admin admin] PATH_INFO: `/v3/projects/553c97c8e1044a318fe94a832a7c9cb9/users/3487f733ec204c5db889531c181bb657/roles/812d3c567c52469cb2b35709fbdc857c` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:55.001234 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c683ccca-1b1c-426d-8b06-e5413024f41b admin admin] RBAC: Authorizing `identity:create_grant(project_id=553c97c8e1044a318fe94a832a7c9cb9, user_id=3487f733ec204c5db889531c181bb657, role_id=812d3c567c52469cb2b35709fbdc857c)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:01:55.007132 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c683ccca-1b1c-426d-8b06-e5413024f41b admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:01:55.047587 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15745cb1-00fd-414b-babe-2ecfb4b468c0 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:01:55.048715 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 986/1948] 10.222.0.46 () {60 vars in 1482 bytes} [Thu Jun 23 16:01:54 2022] PUT /identity/v3/projects/553c97c8e1044a318fe94a832a7c9cb9/users/3487f733ec204c5db889531c181bb657/roles/812d3c567c52469cb2b35709fbdc857c => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 16:01:55.069638 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15745cb1-00fd-414b-babe-2ecfb4b468c0 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:01:55.174869 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bac8d9d-111e-4e6a-ae47-a158e523bea7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:01:55.176046 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bac8d9d-111e-4e6a-ae47-a158e523bea7 admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:55.176545 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bac8d9d-111e-4e6a-ae47-a158e523bea7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:55.177056 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bac8d9d-111e-4e6a-ae47-a158e523bea7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:55.178539 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bac8d9d-111e-4e6a-ae47-a158e523bea7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:01:55.183237 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bac8d9d-111e-4e6a-ae47-a158e523bea7 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:01:55.211068 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 963/1949] 10.222.0.46 () {58 vars in 1121 bytes} [Thu Jun 23 16:01:55 2022] GET /identity/v3/roles => generated 1420 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 16:01:55.219213 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c683ccca-1b1c-426d-8b06-e5413024f41b admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:01:55.243437 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c683ccca-1b1c-426d-8b06-e5413024f41b admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:01:55.248494 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8247ba5e-aa92-4ac4-a2ca-fb806fbf5527 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:01:55.249448 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8247ba5e-aa92-4ac4-a2ca-fb806fbf5527 admin admin] REQUEST_METHOD: `PUT` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:55.249739 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8247ba5e-aa92-4ac4-a2ca-fb806fbf5527 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:55.250014 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8247ba5e-aa92-4ac4-a2ca-fb806fbf5527 admin admin] PATH_INFO: `/v3/projects/553c97c8e1044a318fe94a832a7c9cb9/users/3487f733ec204c5db889531c181bb657/roles/812d3c567c52469cb2b35709fbdc857c` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:55.252222 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8247ba5e-aa92-4ac4-a2ca-fb806fbf5527 admin admin] RBAC: Authorizing `identity:create_grant(project_id=553c97c8e1044a318fe94a832a7c9cb9, user_id=3487f733ec204c5db889531c181bb657, role_id=812d3c567c52469cb2b35709fbdc857c)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:01:55.255016 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8247ba5e-aa92-4ac4-a2ca-fb806fbf5527 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:01:55.267731 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 987/1950] 10.222.0.46 () {60 vars in 1482 bytes} [Thu Jun 23 16:01:55 2022] PUT /identity/v3/projects/553c97c8e1044a318fe94a832a7c9cb9/users/3487f733ec204c5db889531c181bb657/roles/812d3c567c52469cb2b35709fbdc857c => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 16:01:55.287367 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee36ca4-a9db-4d17-b9c8-dffba435978f None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:55.287756 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee36ca4-a9db-4d17-b9c8-dffba435978f None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:55.288120 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee36ca4-a9db-4d17-b9c8-dffba435978f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:55.319720 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-bee36ca4-a9db-4d17-b9c8-dffba435978f None None] MFA Rules not processed for user `3487f733ec204c5db889531c181bb657`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:01:55.321596 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-bee36ca4-a9db-4d17-b9c8-dffba435978f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:01:55.378647 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:01:55.407455 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 964/1951] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:01:55 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 16:01:55.419819 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8247ba5e-aa92-4ac4-a2ca-fb806fbf5527 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:01:55.421636 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-8247ba5e-aa92-4ac4-a2ca-fb806fbf5527 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:01:55.470603 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:01:55.524732 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8247ba5e-aa92-4ac4-a2ca-fb806fbf5527 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:01:55.532197 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec77a2c-d98b-42d4-b8df-bc69137f885f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce18eaaba3a6477684fff1fd98ec8b51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:01:55.533347 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec77a2c-d98b-42d4-b8df-bc69137f885f service cinder] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:55.533879 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec77a2c-d98b-42d4-b8df-bc69137f885f service cinder] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:55.534352 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec77a2c-d98b-42d4-b8df-bc69137f885f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:55.643276 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec77a2c-d98b-42d4-b8df-bc69137f885f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:01:55.645775 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec77a2c-d98b-42d4-b8df-bc69137f885f service cinder] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:01:55.659821 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 988/1952] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jun 23 16:01:55 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 240 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 16:01:56.561892 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee36ca4-a9db-4d17-b9c8-dffba435978f None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:01:56.564184 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-bee36ca4-a9db-4d17-b9c8-dffba435978f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:01:56.667120 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee36ca4-a9db-4d17-b9c8-dffba435978f None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:01:56.672982 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8fe5f7-bd61-4662-85f8-620a0afdfa9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:01:56.674324 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8fe5f7-bd61-4662-85f8-620a0afdfa9a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:56.674720 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8fe5f7-bd61-4662-85f8-620a0afdfa9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:56.675088 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8fe5f7-bd61-4662-85f8-620a0afdfa9a admin admin] PATH_INFO: `/v3/users/0f1e3a6043ac4b96b5ee27dc88d7f1e1` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:56.676806 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8fe5f7-bd61-4662-85f8-620a0afdfa9a admin admin] RBAC: Authorizing `identity:delete_user(user_id=0f1e3a6043ac4b96b5ee27dc88d7f1e1)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:01:56.679278 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8fe5f7-bd61-4662-85f8-620a0afdfa9a admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:01:56.738305 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-9e8fe5f7-bd61-4662-85f8-620a0afdfa9a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0f1e3a6043ac4b96b5ee27dc88d7f1e1'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:01:56.739512 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-9e8fe5f7-bd61-4662-85f8-620a0afdfa9a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0f1e3a6043ac4b96b5ee27dc88d7f1e1'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:01:56.752072 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-9e8fe5f7-bd61-4662-85f8-620a0afdfa9a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0f1e3a6043ac4b96b5ee27dc88d7f1e1'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:01:56.764786 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-9e8fe5f7-bd61-4662-85f8-620a0afdfa9a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0f1e3a6043ac4b96b5ee27dc88d7f1e1'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:01:56.765661 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-9e8fe5f7-bd61-4662-85f8-620a0afdfa9a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0f1e3a6043ac4b96b5ee27dc88d7f1e1'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:01:56.792237 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 965/1953] 10.222.0.46 () {58 vars in 1223 bytes} [Thu Jun 23 16:01:56 2022] DELETE /identity/v3/users/0f1e3a6043ac4b96b5ee27dc88d7f1e1 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 16:01:56.796801 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8976a2e3-348c-49ae-a2c2-592d070b9c30 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:56.797356 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8976a2e3-348c-49ae-a2c2-592d070b9c30 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:56.797356 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8976a2e3-348c-49ae-a2c2-592d070b9c30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:56.816118 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-8976a2e3-348c-49ae-a2c2-592d070b9c30 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:01:56.817361 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-8976a2e3-348c-49ae-a2c2-592d070b9c30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:01:56.905976 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 989/1954] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:01:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 16:01:56.916548 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8fe5f7-bd61-4662-85f8-620a0afdfa9a admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:01:56.918100 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-9e8fe5f7-bd61-4662-85f8-620a0afdfa9a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:01:57.006750 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8fe5f7-bd61-4662-85f8-620a0afdfa9a admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:01:57.011268 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7c74e5-f2a3-4f42-9109-5a658bd9158f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51784064951a4619ae09b9167fb0fd0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:01:57.011948 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7c74e5-f2a3-4f42-9109-5a658bd9158f service neutron] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:57.012235 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7c74e5-f2a3-4f42-9109-5a658bd9158f service neutron] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:57.012541 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7c74e5-f2a3-4f42-9109-5a658bd9158f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:57.024500 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7c74e5-f2a3-4f42-9109-5a658bd9158f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:01:57.026234 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7c74e5-f2a3-4f42-9109-5a658bd9158f service neutron] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:01:57.043648 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 966/1955] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jun 23 16:01:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 16:01:57.567072 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8976a2e3-348c-49ae-a2c2-592d070b9c30 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:01:57.569317 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-8976a2e3-348c-49ae-a2c2-592d070b9c30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:01:57.610667 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8976a2e3-348c-49ae-a2c2-592d070b9c30 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:01:57.622956 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27edba7-2e60-4432-817c-78e550a21a54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:01:57.625096 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27edba7-2e60-4432-817c-78e550a21a54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:57.627643 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27edba7-2e60-4432-817c-78e550a21a54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:57.628914 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27edba7-2e60-4432-817c-78e550a21a54 admin admin] PATH_INFO: `/v3/projects/ad22a947e7cc4aa8857a06f120c0354e` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:57.631970 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27edba7-2e60-4432-817c-78e550a21a54 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ad22a947e7cc4aa8857a06f120c0354e)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:01:57.633968 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27edba7-2e60-4432-817c-78e550a21a54 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:01:57.718308 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-a27edba7-2e60-4432-817c-78e550a21a54 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ad22a947e7cc4aa8857a06f120c0354e', 'request_id': 'req-a27edba7-2e60-4432-817c-78e550a21a54'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:01:57.727829 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-a27edba7-2e60-4432-817c-78e550a21a54 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ad22a947e7cc4aa8857a06f120c0354e', 'request_id': 'req-a27edba7-2e60-4432-817c-78e550a21a54'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:01:57.738370 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-a27edba7-2e60-4432-817c-78e550a21a54 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ad22a947e7cc4aa8857a06f120c0354e', 'request_id': 'req-a27edba7-2e60-4432-817c-78e550a21a54'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:01:57.763474 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-a27edba7-2e60-4432-817c-78e550a21a54 admin admin] The token cache is being invalidate because project ad22a947e7cc4aa8857a06f120c0354e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80153) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 16:01:57.764027 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-a27edba7-2e60-4432-817c-78e550a21a54 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:01:57.765785 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-a27edba7-2e60-4432-817c-78e550a21a54 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:01:57.771666 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 990/1956] 10.222.0.46 () {58 vars in 1232 bytes} [Thu Jun 23 16:01:57 2022] DELETE /identity/v3/projects/ad22a947e7cc4aa8857a06f120c0354e => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 16:01:57.777234 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7c74e5-f2a3-4f42-9109-5a658bd9158f service neutron] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:01:57.789748 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7c74e5-f2a3-4f42-9109-5a658bd9158f service neutron] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:01:57.796028 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:01:57.890365 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8c3be1-52e2-4687-855b-5c3603ac185d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:01:57.891364 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8c3be1-52e2-4687-855b-5c3603ac185d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:57.891650 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8c3be1-52e2-4687-855b-5c3603ac185d admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:57.892755 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8c3be1-52e2-4687-855b-5c3603ac185d admin admin] PATH_INFO: `/v3/users/100dff85187245af9bcb1eb3f1adc67c` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:57.893398 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8c3be1-52e2-4687-855b-5c3603ac185d admin admin] RBAC: Authorizing `identity:delete_user(user_id=100dff85187245af9bcb1eb3f1adc67c)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:01:57.895492 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8c3be1-52e2-4687-855b-5c3603ac185d admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:01:57.950794 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-3a8c3be1-52e2-4687-855b-5c3603ac185d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '100dff85187245af9bcb1eb3f1adc67c'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:01:57.951469 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-3a8c3be1-52e2-4687-855b-5c3603ac185d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '100dff85187245af9bcb1eb3f1adc67c'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:01:57.959391 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-3a8c3be1-52e2-4687-855b-5c3603ac185d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '100dff85187245af9bcb1eb3f1adc67c'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:01:57.968633 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-3a8c3be1-52e2-4687-855b-5c3603ac185d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '100dff85187245af9bcb1eb3f1adc67c'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:01:57.969363 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-3a8c3be1-52e2-4687-855b-5c3603ac185d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '100dff85187245af9bcb1eb3f1adc67c'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:01:57.990386 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 967/1957] 10.222.0.46 () {58 vars in 1223 bytes} [Thu Jun 23 16:01:57 2022] DELETE /identity/v3/users/100dff85187245af9bcb1eb3f1adc67c => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 16:01:58.247742 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27edba7-2e60-4432-817c-78e550a21a54 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:01:58.249268 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-a27edba7-2e60-4432-817c-78e550a21a54 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:01:58.263351 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27edba7-2e60-4432-817c-78e550a21a54 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:01:58.269547 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:01:58.365664 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206ed3c5-5c1a-4f94-855b-aaa3f10172e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:01:58.367044 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206ed3c5-5c1a-4f94-855b-aaa3f10172e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:58.368234 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206ed3c5-5c1a-4f94-855b-aaa3f10172e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:58.368690 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206ed3c5-5c1a-4f94-855b-aaa3f10172e7 admin admin] PATH_INFO: `/v3/users/661a7520a6de4399b19afecbad7c6164` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:58.370395 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206ed3c5-5c1a-4f94-855b-aaa3f10172e7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=661a7520a6de4399b19afecbad7c6164)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:01:58.372830 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206ed3c5-5c1a-4f94-855b-aaa3f10172e7 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:01:58.442984 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8c3be1-52e2-4687-855b-5c3603ac185d admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:01:58.446205 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-3a8c3be1-52e2-4687-855b-5c3603ac185d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:01:58.475350 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8c3be1-52e2-4687-855b-5c3603ac185d admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:01:58.482095 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-206ed3c5-5c1a-4f94-855b-aaa3f10172e7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '661a7520a6de4399b19afecbad7c6164'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:01:58.484523 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4912877a-3419-4764-9417-1cc43485d6b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:01:58.485844 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4912877a-3419-4764-9417-1cc43485d6b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:58.486351 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4912877a-3419-4764-9417-1cc43485d6b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:58.486973 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4912877a-3419-4764-9417-1cc43485d6b2 admin admin] PATH_INFO: `/v3/projects/a2c187698a23468cafa0271182693783` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:58.489356 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4912877a-3419-4764-9417-1cc43485d6b2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a2c187698a23468cafa0271182693783)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:01:58.490548 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-206ed3c5-5c1a-4f94-855b-aaa3f10172e7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '661a7520a6de4399b19afecbad7c6164'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:01:58.491332 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-206ed3c5-5c1a-4f94-855b-aaa3f10172e7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '661a7520a6de4399b19afecbad7c6164'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:01:58.491951 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-206ed3c5-5c1a-4f94-855b-aaa3f10172e7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '661a7520a6de4399b19afecbad7c6164'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:01:58.496440 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4912877a-3419-4764-9417-1cc43485d6b2 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:01:58.507208 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-206ed3c5-5c1a-4f94-855b-aaa3f10172e7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '661a7520a6de4399b19afecbad7c6164'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:01:58.524366 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 991/1958] 10.222.0.46 () {58 vars in 1223 bytes} [Thu Jun 23 16:01:58 2022] DELETE /identity/v3/users/661a7520a6de4399b19afecbad7c6164 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 16:01:58.529616 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6225b21-701b-44d6-a029-702bd2728af8 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:58.529983 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6225b21-701b-44d6-a029-702bd2728af8 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:58.530317 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6225b21-701b-44d6-a029-702bd2728af8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:58.552574 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-c6225b21-701b-44d6-a029-702bd2728af8 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:01:58.553850 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-c6225b21-701b-44d6-a029-702bd2728af8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:01:58.601581 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-4912877a-3419-4764-9417-1cc43485d6b2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a2c187698a23468cafa0271182693783', 'request_id': 'req-4912877a-3419-4764-9417-1cc43485d6b2'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:01:58.612782 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-4912877a-3419-4764-9417-1cc43485d6b2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a2c187698a23468cafa0271182693783', 'request_id': 'req-4912877a-3419-4764-9417-1cc43485d6b2'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:01:58.625675 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-4912877a-3419-4764-9417-1cc43485d6b2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a2c187698a23468cafa0271182693783', 'request_id': 'req-4912877a-3419-4764-9417-1cc43485d6b2'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:01:58.648304 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-4912877a-3419-4764-9417-1cc43485d6b2 admin admin] The token cache is being invalidate because project a2c187698a23468cafa0271182693783 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80154) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 16:01:58.648601 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-4912877a-3419-4764-9417-1cc43485d6b2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:01:58.649339 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-4912877a-3419-4764-9417-1cc43485d6b2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:01:58.654725 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 968/1959] 10.222.0.46 () {58 vars in 1232 bytes} [Thu Jun 23 16:01:58 2022] DELETE /identity/v3/projects/a2c187698a23468cafa0271182693783 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 16:01:58.656587 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db17e75d-0e38-468d-9b88-809ce4f71a14 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:58.657008 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db17e75d-0e38-468d-9b88-809ce4f71a14 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:58.657375 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db17e75d-0e38-468d-9b88-809ce4f71a14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:58.670057 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 992/1960] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:01:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 16:01:58.680809 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6225b21-701b-44d6-a029-702bd2728af8 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:01:58.682375 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-c6225b21-701b-44d6-a029-702bd2728af8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:01:58.684395 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-db17e75d-0e38-468d-9b88-809ce4f71a14 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:01:58.685621 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-db17e75d-0e38-468d-9b88-809ce4f71a14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:01:58.691933 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:01:58.731332 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:01:58.778053 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 969/1961] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:01:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 16:01:58.791233 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22fbc3d-0b5a-4664-b16f-a0da45f97319 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:58.791532 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22fbc3d-0b5a-4664-b16f-a0da45f97319 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:58.791797 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22fbc3d-0b5a-4664-b16f-a0da45f97319 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:58.803072 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6225b21-701b-44d6-a029-702bd2728af8 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:01:58.811749 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-e22fbc3d-0b5a-4664-b16f-a0da45f97319 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:01:58.813365 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-e22fbc3d-0b5a-4664-b16f-a0da45f97319 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:01:58.822720 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 970/1962] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:01:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 16:01:58.827111 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22fbc3d-0b5a-4664-b16f-a0da45f97319 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:01:58.841175 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22fbc3d-0b5a-4664-b16f-a0da45f97319 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:01:58.846725 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b72180-582a-4fa5-9285-c4a7e1b5e2da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:01:58.847450 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b72180-582a-4fa5-9285-c4a7e1b5e2da admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:58.847740 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b72180-582a-4fa5-9285-c4a7e1b5e2da admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:58.848056 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b72180-582a-4fa5-9285-c4a7e1b5e2da admin admin] PATH_INFO: `/v3/domains` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:58.848830 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b72180-582a-4fa5-9285-c4a7e1b5e2da admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 16:01:58.849145 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b72180-582a-4fa5-9285-c4a7e1b5e2da admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:01:58.850847 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b72180-582a-4fa5-9285-c4a7e1b5e2da admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:01:58.864617 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 971/1963] 10.222.0.46 () {58 vars in 1152 bytes} [Thu Jun 23 16:01:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 16:01:58.869758 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b72180-582a-4fa5-9285-c4a7e1b5e2da admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:01:58.882876 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b72180-582a-4fa5-9285-c4a7e1b5e2da admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:01:58.886120 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cc77a0-09ec-4e20-8c72-400dca9efcf3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51784064951a4619ae09b9167fb0fd0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:01:58.886925 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cc77a0-09ec-4e20-8c72-400dca9efcf3 service neutron] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:58.887365 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cc77a0-09ec-4e20-8c72-400dca9efcf3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:58.887650 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cc77a0-09ec-4e20-8c72-400dca9efcf3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:58.891693 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef573955-cd16-46b4-b0b8-7694cc1b956f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:01:58.893726 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef573955-cd16-46b4-b0b8-7694cc1b956f admin admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:58.893726 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef573955-cd16-46b4-b0b8-7694cc1b956f admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:58.893726 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef573955-cd16-46b4-b0b8-7694cc1b956f admin admin] PATH_INFO: `/v3/projects` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:58.894579 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef573955-cd16-46b4-b0b8-7694cc1b956f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:01:58.901589 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef573955-cd16-46b4-b0b8-7694cc1b956f admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:01:58.915794 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94cc77a0-09ec-4e20-8c72-400dca9efcf3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:01:58.917160 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94cc77a0-09ec-4e20-8c72-400dca9efcf3 service neutron] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:01:58.930316 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 993/1964] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jun 23 16:01:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 251 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 16:01:58.939160 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 972/1965] 10.222.0.46 () {60 vars in 1152 bytes} [Thu Jun 23 16:01:58 2022] POST /identity/v3/projects => generated 409 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 16:01:58.945740 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cc77a0-09ec-4e20-8c72-400dca9efcf3 service neutron] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:01:58.959791 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cc77a0-09ec-4e20-8c72-400dca9efcf3 service neutron] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:01:59.047945 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafb5bc7-2804-4865-80bd-e39c0540b57d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:01:59.048639 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafb5bc7-2804-4865-80bd-e39c0540b57d admin admin] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:59.049158 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafb5bc7-2804-4865-80bd-e39c0540b57d admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:59.050083 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafb5bc7-2804-4865-80bd-e39c0540b57d admin admin] PATH_INFO: `/v3/users` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:59.050272 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafb5bc7-2804-4865-80bd-e39c0540b57d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:01:59.052492 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafb5bc7-2804-4865-80bd-e39c0540b57d admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:01:59.085358 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 994/1966] 10.222.0.46 () {60 vars in 1143 bytes} [Thu Jun 23 16:01:58 2022] POST /identity/v3/users => generated 305 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 16:01:59.089904 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef573955-cd16-46b4-b0b8-7694cc1b956f admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:01:59.101713 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef573955-cd16-46b4-b0b8-7694cc1b956f admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:01:59.107439 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c47c9f1-de8d-4393-b548-3bf6278634f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:01:59.108255 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c47c9f1-de8d-4393-b548-3bf6278634f3 admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:59.108642 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c47c9f1-de8d-4393-b548-3bf6278634f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:59.109045 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c47c9f1-de8d-4393-b548-3bf6278634f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:59.109933 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c47c9f1-de8d-4393-b548-3bf6278634f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:01:59.111987 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c47c9f1-de8d-4393-b548-3bf6278634f3 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:01:59.126624 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 973/1967] 10.222.0.46 () {58 vars in 1121 bytes} [Thu Jun 23 16:01:59 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 16:01:59.128164 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafb5bc7-2804-4865-80bd-e39c0540b57d admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:01:59.138860 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafb5bc7-2804-4865-80bd-e39c0540b57d admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:01:59.144144 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6014b9-90c2-4a4c-8688-557cbf3d5e88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:01:59.144840 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6014b9-90c2-4a4c-8688-557cbf3d5e88 admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:59.145123 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6014b9-90c2-4a4c-8688-557cbf3d5e88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:59.145395 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6014b9-90c2-4a4c-8688-557cbf3d5e88 admin admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:59.146141 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6014b9-90c2-4a4c-8688-557cbf3d5e88 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:01:59.147807 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6014b9-90c2-4a4c-8688-557cbf3d5e88 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:01:59.162659 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 995/1968] 10.222.0.46 () {58 vars in 1121 bytes} [Thu Jun 23 16:01:59 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 16:01:59.165092 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c47c9f1-de8d-4393-b548-3bf6278634f3 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:01:59.176436 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c47c9f1-de8d-4393-b548-3bf6278634f3 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:01:59.182399 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ec8a97-1b0d-48f7-bc47-e48e5e2477af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:01:59.183609 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ec8a97-1b0d-48f7-bc47-e48e5e2477af admin admin] REQUEST_METHOD: `PUT` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:59.184089 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ec8a97-1b0d-48f7-bc47-e48e5e2477af admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:59.184621 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ec8a97-1b0d-48f7-bc47-e48e5e2477af admin admin] PATH_INFO: `/v3/projects/8387833e3174458a970ca46eedcf0918/users/c4d45077e6814b0e9b1391d98fec2678/roles/e2a5bbf3200a4d4091d72bc7ff78c34d` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:59.202841 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ec8a97-1b0d-48f7-bc47-e48e5e2477af admin admin] RBAC: Authorizing `identity:create_grant(project_id=8387833e3174458a970ca46eedcf0918, user_id=c4d45077e6814b0e9b1391d98fec2678, role_id=e2a5bbf3200a4d4091d72bc7ff78c34d)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:01:59.209839 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ec8a97-1b0d-48f7-bc47-e48e5e2477af admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:01:59.228723 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 974/1969] 10.222.0.46 () {60 vars in 1482 bytes} [Thu Jun 23 16:01:59 2022] PUT /identity/v3/projects/8387833e3174458a970ca46eedcf0918/users/c4d45077e6814b0e9b1391d98fec2678/roles/e2a5bbf3200a4d4091d72bc7ff78c34d => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 16:01:59.239584 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0918c256-ab33-4438-bac9-ec24fa592b6a None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:59.240236 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0918c256-ab33-4438-bac9-ec24fa592b6a None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:59.240236 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0918c256-ab33-4438-bac9-ec24fa592b6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:59.275109 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-0918c256-ab33-4438-bac9-ec24fa592b6a None None] MFA Rules not processed for user `c4d45077e6814b0e9b1391d98fec2678`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:01:59.276701 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-0918c256-ab33-4438-bac9-ec24fa592b6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:01:59.347141 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:01:59.365462 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 996/1970] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:01:59 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 16:01:59.370329 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ec8a97-1b0d-48f7-bc47-e48e5e2477af admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:01:59.405233 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0918c256-ab33-4438-bac9-ec24fa592b6a None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:01:59.408563 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-0918c256-ab33-4438-bac9-ec24fa592b6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:01:59.502260 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ec8a97-1b0d-48f7-bc47-e48e5e2477af admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:01:59.510903 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afee5ae1-0dba-4a98-aaed-b36ccd8a1dbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:01:59.511725 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afee5ae1-0dba-4a98-aaed-b36ccd8a1dbd admin admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:59.512200 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afee5ae1-0dba-4a98-aaed-b36ccd8a1dbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:59.512880 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afee5ae1-0dba-4a98-aaed-b36ccd8a1dbd admin admin] PATH_INFO: `/v3/projects` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:59.514173 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afee5ae1-0dba-4a98-aaed-b36ccd8a1dbd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:01:59.518007 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afee5ae1-0dba-4a98-aaed-b36ccd8a1dbd admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:01:59.542514 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0918c256-ab33-4438-bac9-ec24fa592b6a None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:01:59.550008 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ead66d-ddc8-4aa6-a97f-aef6881171d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:01:59.550992 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ead66d-ddc8-4aa6-a97f-aef6881171d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:59.551291 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ead66d-ddc8-4aa6-a97f-aef6881171d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:59.551545 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ead66d-ddc8-4aa6-a97f-aef6881171d9 admin admin] PATH_INFO: `/v3/projects/5f09d9a69e08481cb299bdcfdf6efca4` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:59.552961 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ead66d-ddc8-4aa6-a97f-aef6881171d9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5f09d9a69e08481cb299bdcfdf6efca4)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:01:59.555355 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ead66d-ddc8-4aa6-a97f-aef6881171d9 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:01:59.609454 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 975/1971] 10.222.0.46 () {60 vars in 1152 bytes} [Thu Jun 23 16:01:59 2022] POST /identity/v3/projects => generated 407 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 16:01:59.613520 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afee5ae1-0dba-4a98-aaed-b36ccd8a1dbd admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:01:59.646873 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-f4ead66d-ddc8-4aa6-a97f-aef6881171d9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5f09d9a69e08481cb299bdcfdf6efca4', 'request_id': 'req-f4ead66d-ddc8-4aa6-a97f-aef6881171d9'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:01:59.647956 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:01:59.654542 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-f4ead66d-ddc8-4aa6-a97f-aef6881171d9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5f09d9a69e08481cb299bdcfdf6efca4', 'request_id': 'req-f4ead66d-ddc8-4aa6-a97f-aef6881171d9'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:01:59.666963 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-f4ead66d-ddc8-4aa6-a97f-aef6881171d9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5f09d9a69e08481cb299bdcfdf6efca4', 'request_id': 'req-f4ead66d-ddc8-4aa6-a97f-aef6881171d9'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:01:59.689299 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-f4ead66d-ddc8-4aa6-a97f-aef6881171d9 admin admin] The token cache is being invalidate because project 5f09d9a69e08481cb299bdcfdf6efca4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80153) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 16:01:59.689641 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-f4ead66d-ddc8-4aa6-a97f-aef6881171d9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:01:59.690349 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-f4ead66d-ddc8-4aa6-a97f-aef6881171d9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:01:59.692599 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 997/1972] 10.222.0.46 () {58 vars in 1232 bytes} [Thu Jun 23 16:01:59 2022] DELETE /identity/v3/projects/5f09d9a69e08481cb299bdcfdf6efca4 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 16:01:59.699699 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ae8aaf-ec73-4fee-8700-80191b742301 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:59.700137 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ae8aaf-ec73-4fee-8700-80191b742301 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:59.700514 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ae8aaf-ec73-4fee-8700-80191b742301 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:59.724638 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afee5ae1-0dba-4a98-aaed-b36ccd8a1dbd admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:01:59.728063 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-30ae8aaf-ec73-4fee-8700-80191b742301 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:01:59.730047 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-30ae8aaf-ec73-4fee-8700-80191b742301 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:01:59.812836 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42f11b9-dcba-4b76-8b9f-19eba968c3d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:01:59.813861 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42f11b9-dcba-4b76-8b9f-19eba968c3d0 admin admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:59.814262 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42f11b9-dcba-4b76-8b9f-19eba968c3d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:59.814734 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42f11b9-dcba-4b76-8b9f-19eba968c3d0 admin admin] PATH_INFO: `/v3/users` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:59.815559 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42f11b9-dcba-4b76-8b9f-19eba968c3d0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:01:59.818243 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42f11b9-dcba-4b76-8b9f-19eba968c3d0 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:01:59.858830 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 998/1973] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:01:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 16:01:59.861328 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 976/1974] 10.222.0.46 () {60 vars in 1143 bytes} [Thu Jun 23 16:01:59 2022] POST /identity/v3/users => generated 310 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 16:01:59.868421 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42f11b9-dcba-4b76-8b9f-19eba968c3d0 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:01:59.869091 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055dfdbd-3cc2-4294-b96b-394a3d4d9421 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:59.869395 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055dfdbd-3cc2-4294-b96b-394a3d4d9421 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:59.869931 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055dfdbd-3cc2-4294-b96b-394a3d4d9421 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:59.871516 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-f42f11b9-dcba-4b76-8b9f-19eba968c3d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:01:59.892889 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42f11b9-dcba-4b76-8b9f-19eba968c3d0 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:01:59.896217 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-055dfdbd-3cc2-4294-b96b-394a3d4d9421 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:01:59.898809 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-055dfdbd-3cc2-4294-b96b-394a3d4d9421 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:01:59.899491 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12feb5c9-0171-44a6-af5b-e00a0dc0b1b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:01:59.900454 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12feb5c9-0171-44a6-af5b-e00a0dc0b1b1 admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:59.900752 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12feb5c9-0171-44a6-af5b-e00a0dc0b1b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:59.901035 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12feb5c9-0171-44a6-af5b-e00a0dc0b1b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:59.901954 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12feb5c9-0171-44a6-af5b-e00a0dc0b1b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:01:59.903682 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12feb5c9-0171-44a6-af5b-e00a0dc0b1b1 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:01:59.917378 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 999/1975] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:01:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 16:01:59.917651 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 977/1976] 10.222.0.46 () {58 vars in 1121 bytes} [Thu Jun 23 16:01:59 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 16:01:59.920640 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055dfdbd-3cc2-4294-b96b-394a3d4d9421 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:01:59.924339 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12feb5c9-0171-44a6-af5b-e00a0dc0b1b1 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:01:59.933284 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055dfdbd-3cc2-4294-b96b-394a3d4d9421 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:01:59.940343 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83dd0d4f-99cf-4178-8f6a-417441b940ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:01:59.941238 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dd0d4f-99cf-4178-8f6a-417441b940ee admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:59.941666 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dd0d4f-99cf-4178-8f6a-417441b940ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:59.942025 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dd0d4f-99cf-4178-8f6a-417441b940ee admin admin] PATH_INFO: `/v3/domains` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:59.942627 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12feb5c9-0171-44a6-af5b-e00a0dc0b1b1 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:01:59.942849 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83dd0d4f-99cf-4178-8f6a-417441b940ee admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 16:01:59.943179 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83dd0d4f-99cf-4178-8f6a-417441b940ee admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:01:59.946980 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83dd0d4f-99cf-4178-8f6a-417441b940ee admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:01:59.953631 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5fd81b-801e-4202-bb91-318cb2e99713 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:01:59.954846 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5fd81b-801e-4202-bb91-318cb2e99713 admin admin] REQUEST_METHOD: `PUT` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:01:59.955220 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5fd81b-801e-4202-bb91-318cb2e99713 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:01:59.955699 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5fd81b-801e-4202-bb91-318cb2e99713 admin admin] PATH_INFO: `/v3/projects/b7b8418309e64dd09fe564bfada4e1f8/users/c7e1fe2c351a446db003d57630f3357b/roles/812d3c567c52469cb2b35709fbdc857c` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:01:59.970426 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1000/1977] 10.222.0.46 () {58 vars in 1152 bytes} [Thu Jun 23 16:01:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 16:01:59.972708 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83dd0d4f-99cf-4178-8f6a-417441b940ee admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:01:59.979016 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5fd81b-801e-4202-bb91-318cb2e99713 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b7b8418309e64dd09fe564bfada4e1f8, user_id=c7e1fe2c351a446db003d57630f3357b, role_id=812d3c567c52469cb2b35709fbdc857c)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:01:59.985304 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5fd81b-801e-4202-bb91-318cb2e99713 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:01:59.991524 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83dd0d4f-99cf-4178-8f6a-417441b940ee admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:02:00.006366 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2c3649-60b2-40ba-9e72-c551e746117f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:02:00.007903 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2c3649-60b2-40ba-9e72-c551e746117f admin admin] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:00.008127 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2c3649-60b2-40ba-9e72-c551e746117f admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:00.008631 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2c3649-60b2-40ba-9e72-c551e746117f admin admin] PATH_INFO: `/v3/projects` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:00.009952 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2c3649-60b2-40ba-9e72-c551e746117f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:02:00.014197 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 978/1978] 10.222.0.46 () {60 vars in 1482 bytes} [Thu Jun 23 16:01:59 2022] PUT /identity/v3/projects/b7b8418309e64dd09fe564bfada4e1f8/users/c7e1fe2c351a446db003d57630f3357b/roles/812d3c567c52469cb2b35709fbdc857c => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 16:02:00.015277 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5fd81b-801e-4202-bb91-318cb2e99713 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:02:00.015679 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2c3649-60b2-40ba-9e72-c551e746117f admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:02:00.026756 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:02:00.080252 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1001/1979] 10.222.0.46 () {60 vars in 1152 bytes} [Thu Jun 23 16:01:59 2022] POST /identity/v3/projects => generated 377 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 16:02:00.081512 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2c3649-60b2-40ba-9e72-c551e746117f admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:02:00.151964 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5fd81b-801e-4202-bb91-318cb2e99713 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:02:00.184528 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2c3649-60b2-40ba-9e72-c551e746117f admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:02:00.189707 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3f540a-7baf-4445-9fc4-740191e5806a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:02:00.190526 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3f540a-7baf-4445-9fc4-740191e5806a admin admin] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:00.190908 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3f540a-7baf-4445-9fc4-740191e5806a admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:00.191128 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:02:00.191269 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3f540a-7baf-4445-9fc4-740191e5806a admin admin] PATH_INFO: `/v3/users` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:00.191880 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3f540a-7baf-4445-9fc4-740191e5806a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:02:00.193860 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3f540a-7baf-4445-9fc4-740191e5806a admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:02:00.222258 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1002/1980] 10.222.0.46 () {60 vars in 1143 bytes} [Thu Jun 23 16:02:00 2022] POST /identity/v3/users => generated 289 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 16:02:00.226476 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3f540a-7baf-4445-9fc4-740191e5806a admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:02:00.234815 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae20b12a-307b-446a-96eb-4d2ee33f1794 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:02:00.235630 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae20b12a-307b-446a-96eb-4d2ee33f1794 admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:00.235967 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae20b12a-307b-446a-96eb-4d2ee33f1794 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:00.236433 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae20b12a-307b-446a-96eb-4d2ee33f1794 admin admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:00.237349 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae20b12a-307b-446a-96eb-4d2ee33f1794 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:02:00.237625 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3f540a-7baf-4445-9fc4-740191e5806a admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:02:00.239597 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae20b12a-307b-446a-96eb-4d2ee33f1794 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:02:00.243587 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e837b61-11e1-4b29-b69d-b31d93f9c391 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:02:00.244722 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e837b61-11e1-4b29-b69d-b31d93f9c391 admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:00.245334 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e837b61-11e1-4b29-b69d-b31d93f9c391 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:00.245928 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e837b61-11e1-4b29-b69d-b31d93f9c391 admin admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:00.247085 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e837b61-11e1-4b29-b69d-b31d93f9c391 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:02:00.248841 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e837b61-11e1-4b29-b69d-b31d93f9c391 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:02:00.252732 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 979/1981] 10.222.0.46 () {58 vars in 1121 bytes} [Thu Jun 23 16:02:00 2022] GET /identity/v3/roles => generated 1420 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 16:02:00.256979 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae20b12a-307b-446a-96eb-4d2ee33f1794 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:02:00.261871 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1003/1982] 10.222.0.46 () {58 vars in 1121 bytes} [Thu Jun 23 16:02:00 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 16:02:00.265935 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e837b61-11e1-4b29-b69d-b31d93f9c391 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:02:00.268012 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae20b12a-307b-446a-96eb-4d2ee33f1794 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:02:00.275956 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1caf163a-9bbd-4801-8416-1c8887024d4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:02:00.277396 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1caf163a-9bbd-4801-8416-1c8887024d4f admin admin] REQUEST_METHOD: `PUT` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:00.277759 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1caf163a-9bbd-4801-8416-1c8887024d4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:00.278149 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1caf163a-9bbd-4801-8416-1c8887024d4f admin admin] PATH_INFO: `/v3/projects/b7b8418309e64dd09fe564bfada4e1f8/users/c7e1fe2c351a446db003d57630f3357b/roles/812d3c567c52469cb2b35709fbdc857c` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:00.280476 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1caf163a-9bbd-4801-8416-1c8887024d4f admin admin] RBAC: Authorizing `identity:create_grant(project_id=b7b8418309e64dd09fe564bfada4e1f8, user_id=c7e1fe2c351a446db003d57630f3357b, role_id=812d3c567c52469cb2b35709fbdc857c)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:02:00.281053 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e837b61-11e1-4b29-b69d-b31d93f9c391 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:02:00.284974 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1caf163a-9bbd-4801-8416-1c8887024d4f admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:02:00.286711 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c270285f-2ceb-4aa1-95cc-765702e1ed56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:02:00.287665 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c270285f-2ceb-4aa1-95cc-765702e1ed56 admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:00.287793 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c270285f-2ceb-4aa1-95cc-765702e1ed56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:00.288211 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c270285f-2ceb-4aa1-95cc-765702e1ed56 admin admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:00.290275 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c270285f-2ceb-4aa1-95cc-765702e1ed56 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:02:00.292656 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c270285f-2ceb-4aa1-95cc-765702e1ed56 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:02:00.309205 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1004/1984] 10.222.0.46 () {58 vars in 1121 bytes} [Thu Jun 23 16:02:00 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 16:02:00.309399 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 980/1984] 10.222.0.46 () {60 vars in 1482 bytes} [Thu Jun 23 16:02:00 2022] PUT /identity/v3/projects/b7b8418309e64dd09fe564bfada4e1f8/users/c7e1fe2c351a446db003d57630f3357b/roles/812d3c567c52469cb2b35709fbdc857c => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 16:02:00.311538 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c270285f-2ceb-4aa1-95cc-765702e1ed56 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:02:00.313465 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3148a788-a906-42c6-a394-c6ed9434911c None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:00.314053 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3148a788-a906-42c6-a394-c6ed9434911c None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:00.314053 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3148a788-a906-42c6-a394-c6ed9434911c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:00.325921 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c270285f-2ceb-4aa1-95cc-765702e1ed56 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:02:00.355282 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-3148a788-a906-42c6-a394-c6ed9434911c None None] MFA Rules not processed for user `c7e1fe2c351a446db003d57630f3357b`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:02:00.357877 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-3148a788-a906-42c6-a394-c6ed9434911c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:02:00.424235 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72300fd-c449-41ee-a7ae-056c11fe66f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:02:00.425133 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72300fd-c449-41ee-a7ae-056c11fe66f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:00.425412 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72300fd-c449-41ee-a7ae-056c11fe66f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:00.425685 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72300fd-c449-41ee-a7ae-056c11fe66f5 admin admin] PATH_INFO: `/v3/projects/6ddc880458ff4defa7e2ee9da419f258/users/329682080b7a40b18cf0d24bbf405934/roles/e2a5bbf3200a4d4091d72bc7ff78c34d` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:00.442984 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72300fd-c449-41ee-a7ae-056c11fe66f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ddc880458ff4defa7e2ee9da419f258, user_id=329682080b7a40b18cf0d24bbf405934, role_id=e2a5bbf3200a4d4091d72bc7ff78c34d)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:02:00.445774 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72300fd-c449-41ee-a7ae-056c11fe66f5 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:02:00.449223 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 981/1985] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:02:00 2022] POST /identity/v3/auth/tokens => generated 3341 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 16:02:00.462568 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3148a788-a906-42c6-a394-c6ed9434911c None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:02:00.464172 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-3148a788-a906-42c6-a394-c6ed9434911c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:02:00.465271 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1005/1986] 10.222.0.46 () {60 vars in 1482 bytes} [Thu Jun 23 16:02:00 2022] PUT /identity/v3/projects/6ddc880458ff4defa7e2ee9da419f258/users/329682080b7a40b18cf0d24bbf405934/roles/e2a5bbf3200a4d4091d72bc7ff78c34d => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 16:02:00.476513 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6172194e-804f-4673-b963-27bb0f3ccaff None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:00.477392 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6172194e-804f-4673-b963-27bb0f3ccaff None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:00.477677 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6172194e-804f-4673-b963-27bb0f3ccaff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:00.506770 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-6172194e-804f-4673-b963-27bb0f3ccaff None None] MFA Rules not processed for user `329682080b7a40b18cf0d24bbf405934`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:02:00.508079 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-6172194e-804f-4673-b963-27bb0f3ccaff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:02:00.538503 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:02:00.543051 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3148a788-a906-42c6-a394-c6ed9434911c None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:02:00.602948 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1006/1987] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:02:00 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 16:02:00.607796 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6172194e-804f-4673-b963-27bb0f3ccaff None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:02:00.636717 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c8eee1-0124-404b-ac50-78b3cb893dba service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce18eaaba3a6477684fff1fd98ec8b51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:02:00.637830 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c8eee1-0124-404b-ac50-78b3cb893dba service cinder] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:00.638304 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c8eee1-0124-404b-ac50-78b3cb893dba service cinder] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:00.638839 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c8eee1-0124-404b-ac50-78b3cb893dba service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:00.648017 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:02:00.718383 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6172194e-804f-4673-b963-27bb0f3ccaff None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:02:00.724237 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c288f31-b119-4ccc-9d6a-02c87117f6a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:02:00.724959 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c288f31-b119-4ccc-9d6a-02c87117f6a9 admin admin] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:00.725313 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c288f31-b119-4ccc-9d6a-02c87117f6a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:00.725590 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c288f31-b119-4ccc-9d6a-02c87117f6a9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:00.726360 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c288f31-b119-4ccc-9d6a-02c87117f6a9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:02:00.729745 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c288f31-b119-4ccc-9d6a-02c87117f6a9 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:02:00.730261 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c8eee1-0124-404b-ac50-78b3cb893dba service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:02:00.732694 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c8eee1-0124-404b-ac50-78b3cb893dba service cinder] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:02:00.747853 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 982/1988] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jun 23 16:02:00 2022] GET /identity/v3/auth/tokens => generated 3341 bytes in 287 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 16:02:00.770100 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1007/1989] 10.222.0.46 () {60 vars in 1152 bytes} [Thu Jun 23 16:02:00 2022] POST /identity/v3/projects => generated 377 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 16:02:00.775072 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c8eee1-0124-404b-ac50-78b3cb893dba service cinder] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:02:00.791721 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c8eee1-0124-404b-ac50-78b3cb893dba service cinder] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:02:00.881895 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e13200-970d-41f0-843e-1fdf238b9dda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:02:00.883444 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e13200-970d-41f0-843e-1fdf238b9dda admin admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:00.883850 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e13200-970d-41f0-843e-1fdf238b9dda admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:00.884144 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e13200-970d-41f0-843e-1fdf238b9dda admin admin] PATH_INFO: `/v3/users` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:00.885088 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e13200-970d-41f0-843e-1fdf238b9dda admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:02:00.887319 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e13200-970d-41f0-843e-1fdf238b9dda admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:02:00.918053 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 983/1990] 10.222.0.46 () {60 vars in 1143 bytes} [Thu Jun 23 16:02:00 2022] POST /identity/v3/users => generated 289 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 16:02:00.925510 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c288f31-b119-4ccc-9d6a-02c87117f6a9 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:02:00.957683 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c288f31-b119-4ccc-9d6a-02c87117f6a9 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:02:00.970870 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fec1ca-5059-41f7-ba0f-88bf98b5e8db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:02:00.972511 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fec1ca-5059-41f7-ba0f-88bf98b5e8db admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:00.972945 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fec1ca-5059-41f7-ba0f-88bf98b5e8db admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:00.973506 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fec1ca-5059-41f7-ba0f-88bf98b5e8db admin admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:00.974506 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fec1ca-5059-41f7-ba0f-88bf98b5e8db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:02:00.977239 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fec1ca-5059-41f7-ba0f-88bf98b5e8db admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:02:01.000917 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e13200-970d-41f0-843e-1fdf238b9dda admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:02:01.002189 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1008/1991] 10.222.0.46 () {58 vars in 1121 bytes} [Thu Jun 23 16:02:00 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 16:02:01.017893 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e13200-970d-41f0-843e-1fdf238b9dda admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:02:01.023418 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa2054a-88ec-4dba-a126-d92579e66cb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:02:01.024722 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa2054a-88ec-4dba-a126-d92579e66cb3 admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:01.025004 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa2054a-88ec-4dba-a126-d92579e66cb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:01.025529 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa2054a-88ec-4dba-a126-d92579e66cb3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:01.026299 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa2054a-88ec-4dba-a126-d92579e66cb3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:02:01.027953 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa2054a-88ec-4dba-a126-d92579e66cb3 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:02:01.043077 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 984/1992] 10.222.0.46 () {58 vars in 1121 bytes} [Thu Jun 23 16:02:00 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 16:02:01.048431 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fec1ca-5059-41f7-ba0f-88bf98b5e8db admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:02:01.069006 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fec1ca-5059-41f7-ba0f-88bf98b5e8db admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:02:01.077476 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a36e0b8-ae79-40dc-914d-8582f2b1d7f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:02:01.078467 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a36e0b8-ae79-40dc-914d-8582f2b1d7f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:01.078857 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a36e0b8-ae79-40dc-914d-8582f2b1d7f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:01.079214 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a36e0b8-ae79-40dc-914d-8582f2b1d7f7 admin admin] PATH_INFO: `/v3/projects/d3f70497554e4dacb1f7d35cb19e409d/users/c8be0a3c70824636bf1235cc1db8b8c8/roles/e2a5bbf3200a4d4091d72bc7ff78c34d` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:01.087048 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa2054a-88ec-4dba-a126-d92579e66cb3 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:02:01.089778 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-3aa2054a-88ec-4dba-a126-d92579e66cb3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:02:01.096829 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a36e0b8-ae79-40dc-914d-8582f2b1d7f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d3f70497554e4dacb1f7d35cb19e409d, user_id=c8be0a3c70824636bf1235cc1db8b8c8, role_id=e2a5bbf3200a4d4091d72bc7ff78c34d)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:02:01.099756 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a36e0b8-ae79-40dc-914d-8582f2b1d7f7 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:02:01.115891 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1009/1993] 10.222.0.46 () {60 vars in 1482 bytes} [Thu Jun 23 16:02:01 2022] PUT /identity/v3/projects/d3f70497554e4dacb1f7d35cb19e409d/users/c8be0a3c70824636bf1235cc1db8b8c8/roles/e2a5bbf3200a4d4091d72bc7ff78c34d => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 16:02:01.127079 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7912424-360c-4850-b3e8-01316b7deb92 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:01.127370 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7912424-360c-4850-b3e8-01316b7deb92 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:01.127700 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7912424-360c-4850-b3e8-01316b7deb92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:01.179078 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-a7912424-360c-4850-b3e8-01316b7deb92 None None] MFA Rules not processed for user `c8be0a3c70824636bf1235cc1db8b8c8`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:02:01.182056 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-a7912424-360c-4850-b3e8-01316b7deb92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:02:01.189204 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa2054a-88ec-4dba-a126-d92579e66cb3 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:02:01.241168 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:02:01.282146 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50e9e06-ebb9-4820-89e5-4bb90d1f9bd8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce18eaaba3a6477684fff1fd98ec8b51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:02:01.282880 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50e9e06-ebb9-4820-89e5-4bb90d1f9bd8 service cinder] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:01.283162 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50e9e06-ebb9-4820-89e5-4bb90d1f9bd8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:01.283614 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50e9e06-ebb9-4820-89e5-4bb90d1f9bd8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:01.285634 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-d50e9e06-ebb9-4820-89e5-4bb90d1f9bd8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:02:01.334736 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1010/1994] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:02:01 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 16:02:01.338353 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7912424-360c-4850-b3e8-01316b7deb92 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:02:01.346185 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:02:01.363962 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50e9e06-ebb9-4820-89e5-4bb90d1f9bd8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:02:01.369245 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50e9e06-ebb9-4820-89e5-4bb90d1f9bd8 service cinder] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:02:01.393322 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 985/1995] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jun 23 16:02:01 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 16:02:01.451776 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7912424-360c-4850-b3e8-01316b7deb92 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:02:01.460724 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eec1109-5caf-4929-bce9-e4b046de96c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:02:01.462108 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eec1109-5caf-4929-bce9-e4b046de96c0 admin admin] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:01.462661 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eec1109-5caf-4929-bce9-e4b046de96c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:01.463217 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eec1109-5caf-4929-bce9-e4b046de96c0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:01.464849 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eec1109-5caf-4929-bce9-e4b046de96c0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:02:01.468994 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eec1109-5caf-4929-bce9-e4b046de96c0 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:02:01.523412 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1011/1996] 10.222.0.46 () {60 vars in 1152 bytes} [Thu Jun 23 16:02:01 2022] POST /identity/v3/projects => generated 377 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 16:02:01.528086 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50e9e06-ebb9-4820-89e5-4bb90d1f9bd8 service cinder] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:02:01.541736 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50e9e06-ebb9-4820-89e5-4bb90d1f9bd8 service cinder] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:02:01.664736 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42dbd37-4ed5-41cd-a562-875dab6850c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:02:01.665750 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42dbd37-4ed5-41cd-a562-875dab6850c1 admin admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:01.666230 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42dbd37-4ed5-41cd-a562-875dab6850c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:01.666848 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42dbd37-4ed5-41cd-a562-875dab6850c1 admin admin] PATH_INFO: `/v3/users` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:01.668256 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42dbd37-4ed5-41cd-a562-875dab6850c1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:02:01.672575 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42dbd37-4ed5-41cd-a562-875dab6850c1 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:02:01.714757 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 986/1997] 10.222.0.46 () {60 vars in 1143 bytes} [Thu Jun 23 16:02:01 2022] POST /identity/v3/users => generated 295 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 16:02:01.718065 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eec1109-5caf-4929-bce9-e4b046de96c0 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:02:01.730646 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eec1109-5caf-4929-bce9-e4b046de96c0 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:02:01.735869 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25718d0-6b6b-464d-8d47-282d0b0725ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:02:01.736515 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25718d0-6b6b-464d-8d47-282d0b0725ec admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:01.736891 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25718d0-6b6b-464d-8d47-282d0b0725ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:01.737057 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25718d0-6b6b-464d-8d47-282d0b0725ec admin admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:01.737812 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25718d0-6b6b-464d-8d47-282d0b0725ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:02:01.739516 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25718d0-6b6b-464d-8d47-282d0b0725ec admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:02:01.752322 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1012/1998] 10.222.0.46 () {58 vars in 1121 bytes} [Thu Jun 23 16:02:01 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 16:02:01.758828 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42dbd37-4ed5-41cd-a562-875dab6850c1 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:02:01.772275 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42dbd37-4ed5-41cd-a562-875dab6850c1 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:02:01.780786 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91dd82bf-6c18-4117-8afa-1cc7f24b1dbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:02:01.782644 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dd82bf-6c18-4117-8afa-1cc7f24b1dbb admin admin] REQUEST_METHOD: `PUT` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:01.783019 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dd82bf-6c18-4117-8afa-1cc7f24b1dbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:01.783509 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dd82bf-6c18-4117-8afa-1cc7f24b1dbb admin admin] PATH_INFO: `/v3/projects/eb6459aaf3794fe5b4bed757bbe3bfbf/users/53351ae192c4491ab543810d4d85937a/roles/812d3c567c52469cb2b35709fbdc857c` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:01.806047 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91dd82bf-6c18-4117-8afa-1cc7f24b1dbb admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb6459aaf3794fe5b4bed757bbe3bfbf, user_id=53351ae192c4491ab543810d4d85937a, role_id=812d3c567c52469cb2b35709fbdc857c)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:02:01.810187 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91dd82bf-6c18-4117-8afa-1cc7f24b1dbb admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:02:01.840593 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 987/1999] 10.222.0.46 () {60 vars in 1482 bytes} [Thu Jun 23 16:02:01 2022] PUT /identity/v3/projects/eb6459aaf3794fe5b4bed757bbe3bfbf/users/53351ae192c4491ab543810d4d85937a/roles/812d3c567c52469cb2b35709fbdc857c => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 16:02:01.846190 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25718d0-6b6b-464d-8d47-282d0b0725ec admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:02:01.866048 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25718d0-6b6b-464d-8d47-282d0b0725ec admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:02:02.023131 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734463a3-ffbd-45fa-89fb-3a013624b355 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:02:02.023860 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734463a3-ffbd-45fa-89fb-3a013624b355 admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:02.024141 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734463a3-ffbd-45fa-89fb-3a013624b355 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:02.024419 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734463a3-ffbd-45fa-89fb-3a013624b355 admin admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:02.025179 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734463a3-ffbd-45fa-89fb-3a013624b355 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:02:02.027489 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734463a3-ffbd-45fa-89fb-3a013624b355 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:02:02.047563 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1013/2000] 10.222.0.46 () {58 vars in 1121 bytes} [Thu Jun 23 16:02:01 2022] GET /identity/v3/roles => generated 1420 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 16:02:02.049367 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91dd82bf-6c18-4117-8afa-1cc7f24b1dbb admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:02:02.062572 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91dd82bf-6c18-4117-8afa-1cc7f24b1dbb admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:02:02.068387 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2ea229-fe5d-49c3-8a6c-632647ce6908 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:02:02.069350 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2ea229-fe5d-49c3-8a6c-632647ce6908 admin admin] REQUEST_METHOD: `PUT` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:02.069634 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2ea229-fe5d-49c3-8a6c-632647ce6908 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:02.069975 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2ea229-fe5d-49c3-8a6c-632647ce6908 admin admin] PATH_INFO: `/v3/projects/eb6459aaf3794fe5b4bed757bbe3bfbf/users/53351ae192c4491ab543810d4d85937a/roles/812d3c567c52469cb2b35709fbdc857c` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:02.072966 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2ea229-fe5d-49c3-8a6c-632647ce6908 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb6459aaf3794fe5b4bed757bbe3bfbf, user_id=53351ae192c4491ab543810d4d85937a, role_id=812d3c567c52469cb2b35709fbdc857c)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:02:02.075860 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2ea229-fe5d-49c3-8a6c-632647ce6908 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:02:02.089176 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 988/2001] 10.222.0.46 () {60 vars in 1482 bytes} [Thu Jun 23 16:02:02 2022] PUT /identity/v3/projects/eb6459aaf3794fe5b4bed757bbe3bfbf/users/53351ae192c4491ab543810d4d85937a/roles/812d3c567c52469cb2b35709fbdc857c => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 16:02:02.109631 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35ec595-b454-41e7-974c-6dcf4984e18b None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:02.110290 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35ec595-b454-41e7-974c-6dcf4984e18b None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:02.110866 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35ec595-b454-41e7-974c-6dcf4984e18b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:02.158972 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-a35ec595-b454-41e7-974c-6dcf4984e18b None None] MFA Rules not processed for user `53351ae192c4491ab543810d4d85937a`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:02:02.160463 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-a35ec595-b454-41e7-974c-6dcf4984e18b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:02:02.246397 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:02:02.251749 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1014/2002] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:02:02 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 16:02:02.265230 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2ea229-fe5d-49c3-8a6c-632647ce6908 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:02:02.327554 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:02:02.345986 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2ea229-fe5d-49c3-8a6c-632647ce6908 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:02:02.351299 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bceca03a-911a-4100-af4b-047a19351e6d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce18eaaba3a6477684fff1fd98ec8b51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:02:02.352790 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bceca03a-911a-4100-af4b-047a19351e6d service cinder] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:02.353182 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bceca03a-911a-4100-af4b-047a19351e6d service cinder] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:02.353657 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bceca03a-911a-4100-af4b-047a19351e6d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:02.469218 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bceca03a-911a-4100-af4b-047a19351e6d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:02:02.472356 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bceca03a-911a-4100-af4b-047a19351e6d service cinder] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:02:02.493131 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 989/2003] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jun 23 16:02:02 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 16:02:03.065850 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35ec595-b454-41e7-974c-6dcf4984e18b None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:02:03.083666 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35ec595-b454-41e7-974c-6dcf4984e18b None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:02:03.091064 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374d5527-7452-4e7c-b721-93d9682e4f2f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce18eaaba3a6477684fff1fd98ec8b51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:02:03.091985 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374d5527-7452-4e7c-b721-93d9682e4f2f service cinder] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:03.092563 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374d5527-7452-4e7c-b721-93d9682e4f2f service cinder] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:03.093023 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374d5527-7452-4e7c-b721-93d9682e4f2f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:03.096183 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-374d5527-7452-4e7c-b721-93d9682e4f2f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:02:03.171370 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374d5527-7452-4e7c-b721-93d9682e4f2f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:02:03.172857 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374d5527-7452-4e7c-b721-93d9682e4f2f service cinder] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:02:03.186585 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1015/2004] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jun 23 16:02:03 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 16:02:03.937786 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bceca03a-911a-4100-af4b-047a19351e6d service cinder] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:02:03.939989 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-bceca03a-911a-4100-af4b-047a19351e6d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:02:04.053464 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bceca03a-911a-4100-af4b-047a19351e6d service cinder] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:02:04.060467 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f67722a-17e7-4279-bde5-f2f37d91e17a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:02:04.061376 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f67722a-17e7-4279-bde5-f2f37d91e17a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:04.062755 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f67722a-17e7-4279-bde5-f2f37d91e17a admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:04.062755 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f67722a-17e7-4279-bde5-f2f37d91e17a admin admin] PATH_INFO: `/v3/users/ad0a2a0ec6644b90bc67a9a47b95d7be` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:04.064760 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f67722a-17e7-4279-bde5-f2f37d91e17a admin admin] RBAC: Authorizing `identity:delete_user(user_id=ad0a2a0ec6644b90bc67a9a47b95d7be)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:02:04.066969 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f67722a-17e7-4279-bde5-f2f37d91e17a admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:02:04.165505 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-1f67722a-17e7-4279-bde5-f2f37d91e17a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ad0a2a0ec6644b90bc67a9a47b95d7be'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:04.166860 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-1f67722a-17e7-4279-bde5-f2f37d91e17a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ad0a2a0ec6644b90bc67a9a47b95d7be'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:04.177027 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-1f67722a-17e7-4279-bde5-f2f37d91e17a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ad0a2a0ec6644b90bc67a9a47b95d7be'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:04.189710 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-1f67722a-17e7-4279-bde5-f2f37d91e17a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ad0a2a0ec6644b90bc67a9a47b95d7be'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:04.190415 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-1f67722a-17e7-4279-bde5-f2f37d91e17a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ad0a2a0ec6644b90bc67a9a47b95d7be'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:04.215916 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 990/2005] 10.222.0.46 () {58 vars in 1223 bytes} [Thu Jun 23 16:02:03 2022] DELETE /identity/v3/users/ad0a2a0ec6644b90bc67a9a47b95d7be => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 16:02:04.221931 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5ada00-8747-4752-90b8-0f4dee8ab2b3 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:04.222947 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5ada00-8747-4752-90b8-0f4dee8ab2b3 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:04.223245 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5ada00-8747-4752-90b8-0f4dee8ab2b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:04.247903 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-1c5ada00-8747-4752-90b8-0f4dee8ab2b3 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:02:04.249546 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-1c5ada00-8747-4752-90b8-0f4dee8ab2b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:02:04.384399 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1016/2006] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:02:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 16:02:04.405437 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f67722a-17e7-4279-bde5-f2f37d91e17a admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:02:04.410742 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-1f67722a-17e7-4279-bde5-f2f37d91e17a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:02:04.482481 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:02:04.574454 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f67722a-17e7-4279-bde5-f2f37d91e17a admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:02:04.579786 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c2b9be-cb5b-44a9-923b-024a949b7c88 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51784064951a4619ae09b9167fb0fd0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:02:04.581271 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c2b9be-cb5b-44a9-923b-024a949b7c88 service neutron] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:04.581271 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c2b9be-cb5b-44a9-923b-024a949b7c88 service neutron] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:04.581532 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c2b9be-cb5b-44a9-923b-024a949b7c88 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:04.595337 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c2b9be-cb5b-44a9-923b-024a949b7c88 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:02:04.597003 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c2b9be-cb5b-44a9-923b-024a949b7c88 service neutron] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:02:04.609760 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 991/2007] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jun 23 16:02:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 16:02:05.012793 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5ada00-8747-4752-90b8-0f4dee8ab2b3 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:02:05.014327 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-1c5ada00-8747-4752-90b8-0f4dee8ab2b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:02:05.028404 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5ada00-8747-4752-90b8-0f4dee8ab2b3 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:02:05.033377 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337b3eb4-9985-4984-9f5f-98d069d72d5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:02:05.034313 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337b3eb4-9985-4984-9f5f-98d069d72d5e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:05.034603 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337b3eb4-9985-4984-9f5f-98d069d72d5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:05.034922 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337b3eb4-9985-4984-9f5f-98d069d72d5e admin admin] PATH_INFO: `/v3/projects/6f6106af202449118ab58cfb0197236b` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:05.036259 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337b3eb4-9985-4984-9f5f-98d069d72d5e admin admin] RBAC: Authorizing `identity:delete_project(project_id=6f6106af202449118ab58cfb0197236b)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:02:05.038647 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337b3eb4-9985-4984-9f5f-98d069d72d5e admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:02:05.180173 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-337b3eb4-9985-4984-9f5f-98d069d72d5e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6f6106af202449118ab58cfb0197236b', 'request_id': 'req-337b3eb4-9985-4984-9f5f-98d069d72d5e'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:05.191769 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-337b3eb4-9985-4984-9f5f-98d069d72d5e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6f6106af202449118ab58cfb0197236b', 'request_id': 'req-337b3eb4-9985-4984-9f5f-98d069d72d5e'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:05.206174 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-337b3eb4-9985-4984-9f5f-98d069d72d5e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6f6106af202449118ab58cfb0197236b', 'request_id': 'req-337b3eb4-9985-4984-9f5f-98d069d72d5e'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:05.236811 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-337b3eb4-9985-4984-9f5f-98d069d72d5e admin admin] The token cache is being invalidate because project 6f6106af202449118ab58cfb0197236b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80153) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 16:02:05.237841 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-337b3eb4-9985-4984-9f5f-98d069d72d5e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:05.238322 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-337b3eb4-9985-4984-9f5f-98d069d72d5e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:05.241143 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1017/2008] 10.222.0.46 () {58 vars in 1232 bytes} [Thu Jun 23 16:02:05 2022] DELETE /identity/v3/projects/6f6106af202449118ab58cfb0197236b => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 16:02:05.247196 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c2b9be-cb5b-44a9-923b-024a949b7c88 service neutron] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:02:05.266541 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c2b9be-cb5b-44a9-923b-024a949b7c88 service neutron] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:02:05.366962 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafd0931-abc4-4b55-9dbe-ef8303c93514 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:02:05.368916 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafd0931-abc4-4b55-9dbe-ef8303c93514 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:05.369450 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafd0931-abc4-4b55-9dbe-ef8303c93514 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:05.370036 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafd0931-abc4-4b55-9dbe-ef8303c93514 admin admin] PATH_INFO: `/v3/users/acf8ceb77ac14478a95db91927a5f699` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:05.372617 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafd0931-abc4-4b55-9dbe-ef8303c93514 admin admin] RBAC: Authorizing `identity:delete_user(user_id=acf8ceb77ac14478a95db91927a5f699)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:02:05.377094 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafd0931-abc4-4b55-9dbe-ef8303c93514 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:02:05.456143 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-dafd0931-abc4-4b55-9dbe-ef8303c93514 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'acf8ceb77ac14478a95db91927a5f699'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:05.457827 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-dafd0931-abc4-4b55-9dbe-ef8303c93514 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'acf8ceb77ac14478a95db91927a5f699'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:05.462311 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337b3eb4-9985-4984-9f5f-98d069d72d5e admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:02:05.464650 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-337b3eb4-9985-4984-9f5f-98d069d72d5e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:02:05.479600 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-dafd0931-abc4-4b55-9dbe-ef8303c93514 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'acf8ceb77ac14478a95db91927a5f699'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:05.500700 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-dafd0931-abc4-4b55-9dbe-ef8303c93514 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'acf8ceb77ac14478a95db91927a5f699'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:05.501750 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-dafd0931-abc4-4b55-9dbe-ef8303c93514 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'acf8ceb77ac14478a95db91927a5f699'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:05.511044 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:02:05.547918 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 992/2009] 10.222.0.46 () {58 vars in 1223 bytes} [Thu Jun 23 16:02:05 2022] DELETE /identity/v3/users/acf8ceb77ac14478a95db91927a5f699 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 16:02:05.595811 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337b3eb4-9985-4984-9f5f-98d069d72d5e admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:02:05.706521 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4e1190-20b0-4ad7-bf97-145b967c0dec service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce18eaaba3a6477684fff1fd98ec8b51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:02:05.707274 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4e1190-20b0-4ad7-bf97-145b967c0dec service cinder] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:05.707596 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4e1190-20b0-4ad7-bf97-145b967c0dec service cinder] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:05.707899 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4e1190-20b0-4ad7-bf97-145b967c0dec service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:05.710087 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-7b4e1190-20b0-4ad7-bf97-145b967c0dec service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:02:05.809745 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4e1190-20b0-4ad7-bf97-145b967c0dec service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:02:05.812539 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4e1190-20b0-4ad7-bf97-145b967c0dec service cinder] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:02:05.830061 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1018/2010] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jun 23 16:02:05 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 16:02:06.218127 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafd0931-abc4-4b55-9dbe-ef8303c93514 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:02:06.220327 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-dafd0931-abc4-4b55-9dbe-ef8303c93514 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:02:06.294028 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:02:06.338366 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafd0931-abc4-4b55-9dbe-ef8303c93514 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:02:06.346882 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5ad5c1-59f4-43ef-ae99-c0ca69e3017b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:02:06.349257 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5ad5c1-59f4-43ef-ae99-c0ca69e3017b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:06.349918 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5ad5c1-59f4-43ef-ae99-c0ca69e3017b admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:06.350878 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5ad5c1-59f4-43ef-ae99-c0ca69e3017b admin admin] PATH_INFO: `/v3/projects/03c8ea51d75b4c22a676dd24bc046195` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:06.354548 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5ad5c1-59f4-43ef-ae99-c0ca69e3017b admin admin] RBAC: Authorizing `identity:delete_project(project_id=03c8ea51d75b4c22a676dd24bc046195)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:02:06.357570 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5ad5c1-59f4-43ef-ae99-c0ca69e3017b admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:02:06.440812 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-df5ad5c1-59f4-43ef-ae99-c0ca69e3017b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '03c8ea51d75b4c22a676dd24bc046195', 'request_id': 'req-df5ad5c1-59f4-43ef-ae99-c0ca69e3017b'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:06.448718 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-df5ad5c1-59f4-43ef-ae99-c0ca69e3017b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '03c8ea51d75b4c22a676dd24bc046195', 'request_id': 'req-df5ad5c1-59f4-43ef-ae99-c0ca69e3017b'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:06.457750 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-df5ad5c1-59f4-43ef-ae99-c0ca69e3017b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '03c8ea51d75b4c22a676dd24bc046195', 'request_id': 'req-df5ad5c1-59f4-43ef-ae99-c0ca69e3017b'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:06.479042 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-df5ad5c1-59f4-43ef-ae99-c0ca69e3017b admin admin] The token cache is being invalidate because project 03c8ea51d75b4c22a676dd24bc046195 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80154) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 16:02:06.479590 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-df5ad5c1-59f4-43ef-ae99-c0ca69e3017b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:06.480694 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-df5ad5c1-59f4-43ef-ae99-c0ca69e3017b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:06.483801 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 993/2011] 10.222.0.46 () {58 vars in 1232 bytes} [Thu Jun 23 16:02:06 2022] DELETE /identity/v3/projects/03c8ea51d75b4c22a676dd24bc046195 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 16:02:06.488659 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac24e2b-6077-49dc-92a5-51ace298a9fb None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:06.489492 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac24e2b-6077-49dc-92a5-51ace298a9fb None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:06.489492 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac24e2b-6077-49dc-92a5-51ace298a9fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:06.511844 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-cac24e2b-6077-49dc-92a5-51ace298a9fb None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:02:06.513361 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-cac24e2b-6077-49dc-92a5-51ace298a9fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:02:06.614969 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1019/2012] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:02:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 16:02:06.627679 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a81083e-d614-49ce-aae0-19c0f38d3253 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:06.627969 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a81083e-d614-49ce-aae0-19c0f38d3253 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:06.628253 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a81083e-d614-49ce-aae0-19c0f38d3253 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:06.650077 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-3a81083e-d614-49ce-aae0-19c0f38d3253 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:02:06.651921 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-3a81083e-d614-49ce-aae0-19c0f38d3253 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:02:06.660548 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 994/2013] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:02:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 16:02:06.665141 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac24e2b-6077-49dc-92a5-51ace298a9fb None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:02:06.675951 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac24e2b-6077-49dc-92a5-51ace298a9fb None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:02:06.681492 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5240b592-f4d8-4c6d-a854-05ada0039f87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:02:06.682192 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5240b592-f4d8-4c6d-a854-05ada0039f87 admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:06.682475 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5240b592-f4d8-4c6d-a854-05ada0039f87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:06.682815 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5240b592-f4d8-4c6d-a854-05ada0039f87 admin admin] PATH_INFO: `/v3/domains` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:06.683572 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5240b592-f4d8-4c6d-a854-05ada0039f87 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 16:02:06.684020 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5240b592-f4d8-4c6d-a854-05ada0039f87 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:02:06.685724 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5240b592-f4d8-4c6d-a854-05ada0039f87 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:02:06.700913 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1020/2014] 10.222.0.46 () {58 vars in 1152 bytes} [Thu Jun 23 16:02:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 16:02:06.705355 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a81083e-d614-49ce-aae0-19c0f38d3253 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:02:06.715477 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a81083e-d614-49ce-aae0-19c0f38d3253 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:02:06.720493 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f598785c-f9d5-4213-8b65-22107f897dc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:02:06.721170 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f598785c-f9d5-4213-8b65-22107f897dc3 admin admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:06.721465 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f598785c-f9d5-4213-8b65-22107f897dc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:06.721746 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f598785c-f9d5-4213-8b65-22107f897dc3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:06.722451 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f598785c-f9d5-4213-8b65-22107f897dc3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:02:06.724447 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f598785c-f9d5-4213-8b65-22107f897dc3 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:02:06.759589 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 995/2015] 10.222.0.46 () {60 vars in 1152 bytes} [Thu Jun 23 16:02:06 2022] POST /identity/v3/projects => generated 367 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 16:02:06.760988 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5240b592-f4d8-4c6d-a854-05ada0039f87 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:02:06.774604 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5240b592-f4d8-4c6d-a854-05ada0039f87 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:02:06.798835 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:02:06.876482 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8544df-6d99-4b05-819e-93a69edc81f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:02:06.877212 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8544df-6d99-4b05-819e-93a69edc81f7 admin admin] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:06.877504 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8544df-6d99-4b05-819e-93a69edc81f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:06.877779 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8544df-6d99-4b05-819e-93a69edc81f7 admin admin] PATH_INFO: `/v3/users` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:06.878541 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8544df-6d99-4b05-819e-93a69edc81f7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:02:06.880714 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8544df-6d99-4b05-819e-93a69edc81f7 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:02:06.917745 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1021/2016] 10.222.0.46 () {60 vars in 1143 bytes} [Thu Jun 23 16:02:06 2022] POST /identity/v3/users => generated 284 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 16:02:06.919588 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f598785c-f9d5-4213-8b65-22107f897dc3 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:02:06.942857 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f598785c-f9d5-4213-8b65-22107f897dc3 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:02:06.952417 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d15f75-d397-4080-9e4d-ed090eee0b08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:02:06.953653 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d15f75-d397-4080-9e4d-ed090eee0b08 admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:06.953929 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d15f75-d397-4080-9e4d-ed090eee0b08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:06.954237 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d15f75-d397-4080-9e4d-ed090eee0b08 admin admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:06.955375 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d15f75-d397-4080-9e4d-ed090eee0b08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:02:06.957344 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d15f75-d397-4080-9e4d-ed090eee0b08 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:02:06.973857 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 996/2017] 10.222.0.46 () {58 vars in 1121 bytes} [Thu Jun 23 16:02:06 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 16:02:06.975438 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8544df-6d99-4b05-819e-93a69edc81f7 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:02:06.991162 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8544df-6d99-4b05-819e-93a69edc81f7 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:02:06.996380 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbcb2b9-e9d2-4660-b02d-ae75a4d61402 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:02:06.997227 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbcb2b9-e9d2-4660-b02d-ae75a4d61402 admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:06.997601 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbcb2b9-e9d2-4660-b02d-ae75a4d61402 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:06.997893 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbcb2b9-e9d2-4660-b02d-ae75a4d61402 admin admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:06.999104 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbcb2b9-e9d2-4660-b02d-ae75a4d61402 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:02:07.001118 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbcb2b9-e9d2-4660-b02d-ae75a4d61402 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:02:07.014569 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1022/2018] 10.222.0.46 () {58 vars in 1121 bytes} [Thu Jun 23 16:02:06 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 16:02:07.019096 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d15f75-d397-4080-9e4d-ed090eee0b08 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:02:07.032575 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d15f75-d397-4080-9e4d-ed090eee0b08 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:02:07.037773 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1eaa83-819c-4dd5-9a31-d43ff741e622 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:02:07.038787 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1eaa83-819c-4dd5-9a31-d43ff741e622 admin admin] REQUEST_METHOD: `PUT` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:07.039121 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1eaa83-819c-4dd5-9a31-d43ff741e622 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:07.039401 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1eaa83-819c-4dd5-9a31-d43ff741e622 admin admin] PATH_INFO: `/v3/projects/7f645a5ceee54ecc8a5877c2f23c8636/users/2e35809673e1458ca9935c9a8be6c193/roles/e2a5bbf3200a4d4091d72bc7ff78c34d` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:07.057309 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1eaa83-819c-4dd5-9a31-d43ff741e622 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f645a5ceee54ecc8a5877c2f23c8636, user_id=2e35809673e1458ca9935c9a8be6c193, role_id=e2a5bbf3200a4d4091d72bc7ff78c34d)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:02:07.060343 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1eaa83-819c-4dd5-9a31-d43ff741e622 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:02:07.077996 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 997/2019] 10.222.0.46 () {60 vars in 1482 bytes} [Thu Jun 23 16:02:07 2022] PUT /identity/v3/projects/7f645a5ceee54ecc8a5877c2f23c8636/users/2e35809673e1458ca9935c9a8be6c193/roles/e2a5bbf3200a4d4091d72bc7ff78c34d => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 16:02:07.098041 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335f6286-3cb9-4c63-ac00-73bc974b2304 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:07.098592 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335f6286-3cb9-4c63-ac00-73bc974b2304 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:07.099114 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335f6286-3cb9-4c63-ac00-73bc974b2304 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:07.152481 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-335f6286-3cb9-4c63-ac00-73bc974b2304 None None] MFA Rules not processed for user `2e35809673e1458ca9935c9a8be6c193`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:02:07.154626 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-335f6286-3cb9-4c63-ac00-73bc974b2304 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:02:07.259361 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1eaa83-819c-4dd5-9a31-d43ff741e622 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:02:07.261566 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-ce1eaa83-819c-4dd5-9a31-d43ff741e622 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:02:07.278454 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1023/2020] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:02:07 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 16:02:07.285671 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335f6286-3cb9-4c63-ac00-73bc974b2304 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:02:07.391295 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1eaa83-819c-4dd5-9a31-d43ff741e622 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:02:07.397798 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991fb52a-c1b0-408f-9552-38c9f06f43f9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce18eaaba3a6477684fff1fd98ec8b51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:02:07.398625 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991fb52a-c1b0-408f-9552-38c9f06f43f9 service cinder] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:07.399315 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991fb52a-c1b0-408f-9552-38c9f06f43f9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:07.399826 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991fb52a-c1b0-408f-9552-38c9f06f43f9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:07.402886 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-991fb52a-c1b0-408f-9552-38c9f06f43f9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:02:07.431158 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:02:07.488042 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335f6286-3cb9-4c63-ac00-73bc974b2304 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:02:07.496665 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754b8b3b-45a0-4ec7-a0c8-61b225c3984d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:02:07.498127 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754b8b3b-45a0-4ec7-a0c8-61b225c3984d admin admin] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:07.498569 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754b8b3b-45a0-4ec7-a0c8-61b225c3984d admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:07.499073 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754b8b3b-45a0-4ec7-a0c8-61b225c3984d admin admin] PATH_INFO: `/v3/projects` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:07.499819 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754b8b3b-45a0-4ec7-a0c8-61b225c3984d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:02:07.502184 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754b8b3b-45a0-4ec7-a0c8-61b225c3984d admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:02:07.508896 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991fb52a-c1b0-408f-9552-38c9f06f43f9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:02:07.510750 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991fb52a-c1b0-408f-9552-38c9f06f43f9 service cinder] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:02:07.524143 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 998/2021] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jun 23 16:02:07 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 266 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 16:02:07.545550 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1024/2022] 10.222.0.46 () {60 vars in 1152 bytes} [Thu Jun 23 16:02:07 2022] POST /identity/v3/projects => generated 369 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 16:02:07.546944 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991fb52a-c1b0-408f-9552-38c9f06f43f9 service cinder] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:02:07.562203 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991fb52a-c1b0-408f-9552-38c9f06f43f9 service cinder] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:02:07.652855 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e41350-52cf-4edf-aa54-897496af417e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:02:07.653691 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e41350-52cf-4edf-aa54-897496af417e admin admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:07.653979 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e41350-52cf-4edf-aa54-897496af417e admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:07.654266 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e41350-52cf-4edf-aa54-897496af417e admin admin] PATH_INFO: `/v3/users` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:07.655050 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e41350-52cf-4edf-aa54-897496af417e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:02:07.657059 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e41350-52cf-4edf-aa54-897496af417e admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:02:07.685746 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 999/2023] 10.222.0.46 () {60 vars in 1143 bytes} [Thu Jun 23 16:02:07 2022] POST /identity/v3/users => generated 291 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 16:02:07.687456 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754b8b3b-45a0-4ec7-a0c8-61b225c3984d admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:02:07.709307 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754b8b3b-45a0-4ec7-a0c8-61b225c3984d admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:02:07.714662 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b46945-5e3f-4ef6-bb72-52ff8e961b90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:02:07.715400 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b46945-5e3f-4ef6-bb72-52ff8e961b90 admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:07.715702 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b46945-5e3f-4ef6-bb72-52ff8e961b90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:07.715981 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b46945-5e3f-4ef6-bb72-52ff8e961b90 admin admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:07.716739 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b46945-5e3f-4ef6-bb72-52ff8e961b90 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:02:07.718922 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b46945-5e3f-4ef6-bb72-52ff8e961b90 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:02:07.722334 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:02:07.735446 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1025/2024] 10.222.0.46 () {58 vars in 1121 bytes} [Thu Jun 23 16:02:07 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 16:02:07.740403 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e41350-52cf-4edf-aa54-897496af417e admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:02:07.760269 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e41350-52cf-4edf-aa54-897496af417e admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:02:07.769059 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf078629-9e2b-46e3-a727-de2fdd10a13d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:02:07.771179 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf078629-9e2b-46e3-a727-de2fdd10a13d admin admin] REQUEST_METHOD: `PUT` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:07.771179 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf078629-9e2b-46e3-a727-de2fdd10a13d admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:07.771179 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf078629-9e2b-46e3-a727-de2fdd10a13d admin admin] PATH_INFO: `/v3/projects/564a27eada4d45ae9dabd50b10c54228/users/55067a8b8290424a8c696448183c6327/roles/812d3c567c52469cb2b35709fbdc857c` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:07.792928 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf078629-9e2b-46e3-a727-de2fdd10a13d admin admin] RBAC: Authorizing `identity:create_grant(project_id=564a27eada4d45ae9dabd50b10c54228, user_id=55067a8b8290424a8c696448183c6327, role_id=812d3c567c52469cb2b35709fbdc857c)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:02:07.795955 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf078629-9e2b-46e3-a727-de2fdd10a13d admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:02:07.816629 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1000/2025] 10.222.0.46 () {60 vars in 1482 bytes} [Thu Jun 23 16:02:07 2022] PUT /identity/v3/projects/564a27eada4d45ae9dabd50b10c54228/users/55067a8b8290424a8c696448183c6327/roles/812d3c567c52469cb2b35709fbdc857c => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 16:02:07.824630 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b46945-5e3f-4ef6-bb72-52ff8e961b90 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:02:07.842364 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b46945-5e3f-4ef6-bb72-52ff8e961b90 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:02:07.916051 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e7c7e0-4d70-4448-846a-67d1f2097c8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:02:07.916773 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e7c7e0-4d70-4448-846a-67d1f2097c8d admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:07.917054 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e7c7e0-4d70-4448-846a-67d1f2097c8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:07.917328 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e7c7e0-4d70-4448-846a-67d1f2097c8d admin admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:07.918260 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e7c7e0-4d70-4448-846a-67d1f2097c8d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:02:07.920331 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e7c7e0-4d70-4448-846a-67d1f2097c8d admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:02:07.933080 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1026/2026] 10.222.0.46 () {58 vars in 1121 bytes} [Thu Jun 23 16:02:07 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 16:02:07.937521 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf078629-9e2b-46e3-a727-de2fdd10a13d admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:02:07.952669 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf078629-9e2b-46e3-a727-de2fdd10a13d admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:02:07.958403 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8816e950-5344-4271-913f-f668c12ae6de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:02:07.959372 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8816e950-5344-4271-913f-f668c12ae6de admin admin] REQUEST_METHOD: `PUT` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:07.959656 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8816e950-5344-4271-913f-f668c12ae6de admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:07.959944 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8816e950-5344-4271-913f-f668c12ae6de admin admin] PATH_INFO: `/v3/projects/564a27eada4d45ae9dabd50b10c54228/users/55067a8b8290424a8c696448183c6327/roles/812d3c567c52469cb2b35709fbdc857c` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:07.962611 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8816e950-5344-4271-913f-f668c12ae6de admin admin] RBAC: Authorizing `identity:create_grant(project_id=564a27eada4d45ae9dabd50b10c54228, user_id=55067a8b8290424a8c696448183c6327, role_id=812d3c567c52469cb2b35709fbdc857c)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:02:07.965454 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8816e950-5344-4271-913f-f668c12ae6de admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:02:07.977979 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1001/2027] 10.222.0.46 () {60 vars in 1482 bytes} [Thu Jun 23 16:02:07 2022] PUT /identity/v3/projects/564a27eada4d45ae9dabd50b10c54228/users/55067a8b8290424a8c696448183c6327/roles/812d3c567c52469cb2b35709fbdc857c => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 16:02:07.995744 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62719174-b39f-40d4-b999-ec973cd95532 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:07.996044 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62719174-b39f-40d4-b999-ec973cd95532 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:07.996335 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62719174-b39f-40d4-b999-ec973cd95532 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:08.028608 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-62719174-b39f-40d4-b999-ec973cd95532 None None] MFA Rules not processed for user `55067a8b8290424a8c696448183c6327`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:02:08.029859 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-62719174-b39f-40d4-b999-ec973cd95532 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:02:08.109206 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1027/2028] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:02:07 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 16:02:08.125850 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8816e950-5344-4271-913f-f668c12ae6de admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:02:08.128551 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-8816e950-5344-4271-913f-f668c12ae6de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:02:08.210287 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8816e950-5344-4271-913f-f668c12ae6de admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:02:08.215085 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55607b68-75b8-4943-bf0c-4ed6ebaf79f9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce18eaaba3a6477684fff1fd98ec8b51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:02:08.215945 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55607b68-75b8-4943-bf0c-4ed6ebaf79f9 service cinder] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:08.216274 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55607b68-75b8-4943-bf0c-4ed6ebaf79f9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:08.216719 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55607b68-75b8-4943-bf0c-4ed6ebaf79f9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:08.227618 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55607b68-75b8-4943-bf0c-4ed6ebaf79f9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:02:08.229073 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55607b68-75b8-4943-bf0c-4ed6ebaf79f9 service cinder] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:02:08.240090 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1002/2029] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jun 23 16:02:08 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 16:02:10.124831 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62719174-b39f-40d4-b999-ec973cd95532 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:02:10.212875 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62719174-b39f-40d4-b999-ec973cd95532 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:02:10.217970 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd105737-3be5-4fe7-8d90-a465fb14434a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:02:10.218908 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd105737-3be5-4fe7-8d90-a465fb14434a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:10.219197 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd105737-3be5-4fe7-8d90-a465fb14434a admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:10.219490 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd105737-3be5-4fe7-8d90-a465fb14434a admin admin] PATH_INFO: `/v3/users/2e35809673e1458ca9935c9a8be6c193` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:10.220760 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd105737-3be5-4fe7-8d90-a465fb14434a admin admin] RBAC: Authorizing `identity:delete_user(user_id=2e35809673e1458ca9935c9a8be6c193)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:02:10.222752 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd105737-3be5-4fe7-8d90-a465fb14434a admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:02:10.252895 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:02:10.297058 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-bd105737-3be5-4fe7-8d90-a465fb14434a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2e35809673e1458ca9935c9a8be6c193'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:10.307866 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-bd105737-3be5-4fe7-8d90-a465fb14434a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2e35809673e1458ca9935c9a8be6c193'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:10.308811 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-bd105737-3be5-4fe7-8d90-a465fb14434a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2e35809673e1458ca9935c9a8be6c193'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:10.309697 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-bd105737-3be5-4fe7-8d90-a465fb14434a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2e35809673e1458ca9935c9a8be6c193'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:10.325641 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-bd105737-3be5-4fe7-8d90-a465fb14434a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2e35809673e1458ca9935c9a8be6c193'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:10.361447 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1028/2030] 10.222.0.46 () {58 vars in 1223 bytes} [Thu Jun 23 16:02:10 2022] DELETE /identity/v3/users/2e35809673e1458ca9935c9a8be6c193 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 16:02:10.370172 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d589d799-4da6-4e1f-b774-758f3b8a6201 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:10.370717 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d589d799-4da6-4e1f-b774-758f3b8a6201 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:10.371131 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d589d799-4da6-4e1f-b774-758f3b8a6201 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:10.392789 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-d589d799-4da6-4e1f-b774-758f3b8a6201 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:02:10.394055 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-d589d799-4da6-4e1f-b774-758f3b8a6201 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:02:10.399575 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:02:10.465498 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1003/2031] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:02:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 16:02:10.475686 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd105737-3be5-4fe7-8d90-a465fb14434a admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:02:10.478716 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-bd105737-3be5-4fe7-8d90-a465fb14434a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:02:10.540385 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd105737-3be5-4fe7-8d90-a465fb14434a admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:02:10.544174 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e503d3-2f3d-43bd-9c5a-2e5bdcbdfc6d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51784064951a4619ae09b9167fb0fd0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:02:10.544870 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e503d3-2f3d-43bd-9c5a-2e5bdcbdfc6d service neutron] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:10.545147 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e503d3-2f3d-43bd-9c5a-2e5bdcbdfc6d service neutron] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:10.545425 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e503d3-2f3d-43bd-9c5a-2e5bdcbdfc6d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:10.556493 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e503d3-2f3d-43bd-9c5a-2e5bdcbdfc6d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:02:10.558005 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e503d3-2f3d-43bd-9c5a-2e5bdcbdfc6d service neutron] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:02:10.568971 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1029/2032] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jun 23 16:02:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 16:02:11.155448 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d589d799-4da6-4e1f-b774-758f3b8a6201 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:02:11.158337 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-d589d799-4da6-4e1f-b774-758f3b8a6201 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:02:11.174282 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d589d799-4da6-4e1f-b774-758f3b8a6201 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:02:11.181143 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fe136c-0fe6-4c9d-b757-8276bf45973c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:02:11.181917 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fe136c-0fe6-4c9d-b757-8276bf45973c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:11.182327 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fe136c-0fe6-4c9d-b757-8276bf45973c admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:11.182719 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fe136c-0fe6-4c9d-b757-8276bf45973c admin admin] PATH_INFO: `/v3/projects/7f645a5ceee54ecc8a5877c2f23c8636` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:11.184174 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fe136c-0fe6-4c9d-b757-8276bf45973c admin admin] RBAC: Authorizing `identity:delete_project(project_id=7f645a5ceee54ecc8a5877c2f23c8636)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:02:11.186232 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fe136c-0fe6-4c9d-b757-8276bf45973c admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:02:11.279785 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-94fe136c-0fe6-4c9d-b757-8276bf45973c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7f645a5ceee54ecc8a5877c2f23c8636', 'request_id': 'req-94fe136c-0fe6-4c9d-b757-8276bf45973c'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:11.301229 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-94fe136c-0fe6-4c9d-b757-8276bf45973c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7f645a5ceee54ecc8a5877c2f23c8636', 'request_id': 'req-94fe136c-0fe6-4c9d-b757-8276bf45973c'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:11.311601 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-94fe136c-0fe6-4c9d-b757-8276bf45973c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7f645a5ceee54ecc8a5877c2f23c8636', 'request_id': 'req-94fe136c-0fe6-4c9d-b757-8276bf45973c'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:11.325971 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-94fe136c-0fe6-4c9d-b757-8276bf45973c admin admin] The token cache is being invalidate because project 7f645a5ceee54ecc8a5877c2f23c8636 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80154) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 16:02:11.326401 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-94fe136c-0fe6-4c9d-b757-8276bf45973c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:11.327258 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-94fe136c-0fe6-4c9d-b757-8276bf45973c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:11.330118 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1004/2033] 10.222.0.46 () {58 vars in 1232 bytes} [Thu Jun 23 16:02:11 2022] DELETE /identity/v3/projects/7f645a5ceee54ecc8a5877c2f23c8636 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 16:02:11.334614 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e503d3-2f3d-43bd-9c5a-2e5bdcbdfc6d service neutron] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:02:11.352630 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e503d3-2f3d-43bd-9c5a-2e5bdcbdfc6d service neutron] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:02:11.436858 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba37b24d-d7f7-4d81-9483-e25dafe14051 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:02:11.437764 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba37b24d-d7f7-4d81-9483-e25dafe14051 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:11.438043 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba37b24d-d7f7-4d81-9483-e25dafe14051 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:11.438317 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba37b24d-d7f7-4d81-9483-e25dafe14051 admin admin] PATH_INFO: `/v3/users/55067a8b8290424a8c696448183c6327` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:11.439619 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba37b24d-d7f7-4d81-9483-e25dafe14051 admin admin] RBAC: Authorizing `identity:delete_user(user_id=55067a8b8290424a8c696448183c6327)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:02:11.441725 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba37b24d-d7f7-4d81-9483-e25dafe14051 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:02:11.504938 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:02:11.508245 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-ba37b24d-d7f7-4d81-9483-e25dafe14051 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '55067a8b8290424a8c696448183c6327'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:11.529563 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-ba37b24d-d7f7-4d81-9483-e25dafe14051 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '55067a8b8290424a8c696448183c6327'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:11.530768 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-ba37b24d-d7f7-4d81-9483-e25dafe14051 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '55067a8b8290424a8c696448183c6327'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:11.531796 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-ba37b24d-d7f7-4d81-9483-e25dafe14051 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '55067a8b8290424a8c696448183c6327'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:11.547517 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-ba37b24d-d7f7-4d81-9483-e25dafe14051 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '55067a8b8290424a8c696448183c6327'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:11.569535 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1030/2034] 10.222.0.46 () {58 vars in 1223 bytes} [Thu Jun 23 16:02:11 2022] DELETE /identity/v3/users/55067a8b8290424a8c696448183c6327 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 16:02:12.167126 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fe136c-0fe6-4c9d-b757-8276bf45973c admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:02:12.168653 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-94fe136c-0fe6-4c9d-b757-8276bf45973c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:02:12.181884 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fe136c-0fe6-4c9d-b757-8276bf45973c admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:02:12.249019 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f031a8c3-5462-410a-b235-99681eb9034e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:02:12.251058 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f031a8c3-5462-410a-b235-99681eb9034e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:12.251539 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f031a8c3-5462-410a-b235-99681eb9034e admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:12.251832 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f031a8c3-5462-410a-b235-99681eb9034e admin admin] PATH_INFO: `/v3/projects/564a27eada4d45ae9dabd50b10c54228` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:12.253566 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f031a8c3-5462-410a-b235-99681eb9034e admin admin] RBAC: Authorizing `identity:delete_project(project_id=564a27eada4d45ae9dabd50b10c54228)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:02:12.255846 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f031a8c3-5462-410a-b235-99681eb9034e admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:02:12.311383 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:02:12.322939 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-f031a8c3-5462-410a-b235-99681eb9034e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '564a27eada4d45ae9dabd50b10c54228', 'request_id': 'req-f031a8c3-5462-410a-b235-99681eb9034e'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:12.329067 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-f031a8c3-5462-410a-b235-99681eb9034e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '564a27eada4d45ae9dabd50b10c54228', 'request_id': 'req-f031a8c3-5462-410a-b235-99681eb9034e'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:12.339985 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-f031a8c3-5462-410a-b235-99681eb9034e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '564a27eada4d45ae9dabd50b10c54228', 'request_id': 'req-f031a8c3-5462-410a-b235-99681eb9034e'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:12.353064 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-f031a8c3-5462-410a-b235-99681eb9034e admin admin] The token cache is being invalidate because project 564a27eada4d45ae9dabd50b10c54228 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80154) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 16:02:12.353434 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-f031a8c3-5462-410a-b235-99681eb9034e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:12.354165 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-f031a8c3-5462-410a-b235-99681eb9034e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:12.355834 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1005/2035] 10.222.0.46 () {58 vars in 1232 bytes} [Thu Jun 23 16:02:12 2022] DELETE /identity/v3/projects/564a27eada4d45ae9dabd50b10c54228 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 16:02:12.360762 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deaccc3b-444d-4331-b305-bf19239ec1f9 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:12.361271 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deaccc3b-444d-4331-b305-bf19239ec1f9 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:12.361477 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deaccc3b-444d-4331-b305-bf19239ec1f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:12.381940 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-deaccc3b-444d-4331-b305-bf19239ec1f9 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:02:12.381940 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-deaccc3b-444d-4331-b305-bf19239ec1f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:02:12.468464 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1031/2036] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:02:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 16:02:12.478564 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806b26c9-3b72-4638-92d2-7f1f918e6c4d None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:12.478994 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806b26c9-3b72-4638-92d2-7f1f918e6c4d None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:12.480003 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806b26c9-3b72-4638-92d2-7f1f918e6c4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:12.501276 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-806b26c9-3b72-4638-92d2-7f1f918e6c4d None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:02:12.502591 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-806b26c9-3b72-4638-92d2-7f1f918e6c4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:02:12.511248 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1006/2037] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:02:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 16:02:12.515261 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deaccc3b-444d-4331-b305-bf19239ec1f9 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:02:12.526300 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deaccc3b-444d-4331-b305-bf19239ec1f9 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:02:12.533879 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f75854-bf6b-4864-8304-965dcefeee27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:02:12.534927 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f75854-bf6b-4864-8304-965dcefeee27 admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:12.535393 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f75854-bf6b-4864-8304-965dcefeee27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:12.543173 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f75854-bf6b-4864-8304-965dcefeee27 admin admin] PATH_INFO: `/v3/domains` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:12.544415 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f75854-bf6b-4864-8304-965dcefeee27 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 16:02:12.544749 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f75854-bf6b-4864-8304-965dcefeee27 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:02:12.547005 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f75854-bf6b-4864-8304-965dcefeee27 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:02:12.564988 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1032/2038] 10.222.0.46 () {58 vars in 1152 bytes} [Thu Jun 23 16:02:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 16:02:12.569685 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806b26c9-3b72-4638-92d2-7f1f918e6c4d None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:02:12.580571 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806b26c9-3b72-4638-92d2-7f1f918e6c4d None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:02:12.586338 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170f7148-05c7-4d55-ba4d-83b18aff89bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:02:12.587101 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170f7148-05c7-4d55-ba4d-83b18aff89bf admin admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:12.587386 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170f7148-05c7-4d55-ba4d-83b18aff89bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:12.587806 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170f7148-05c7-4d55-ba4d-83b18aff89bf admin admin] PATH_INFO: `/v3/projects` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:12.588577 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170f7148-05c7-4d55-ba4d-83b18aff89bf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:02:12.590578 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170f7148-05c7-4d55-ba4d-83b18aff89bf admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:02:12.632163 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1007/2039] 10.222.0.46 () {60 vars in 1152 bytes} [Thu Jun 23 16:02:12 2022] POST /identity/v3/projects => generated 385 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 16:02:12.634355 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f75854-bf6b-4864-8304-965dcefeee27 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:02:12.646890 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f75854-bf6b-4864-8304-965dcefeee27 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:02:12.719240 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfdd251-f0d5-40c1-bfe5-462e235b917e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:02:12.719943 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfdd251-f0d5-40c1-bfe5-462e235b917e admin admin] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:12.720230 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfdd251-f0d5-40c1-bfe5-462e235b917e admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:12.720507 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfdd251-f0d5-40c1-bfe5-462e235b917e admin admin] PATH_INFO: `/v3/users` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:12.721243 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfdd251-f0d5-40c1-bfe5-462e235b917e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:02:12.723386 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfdd251-f0d5-40c1-bfe5-462e235b917e admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:02:12.747558 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1033/2040] 10.222.0.46 () {60 vars in 1143 bytes} [Thu Jun 23 16:02:12 2022] POST /identity/v3/users => generated 293 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 16:02:12.748780 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170f7148-05c7-4d55-ba4d-83b18aff89bf admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:02:12.759781 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170f7148-05c7-4d55-ba4d-83b18aff89bf admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:02:12.764787 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9470510c-5498-437a-9909-f74e35a724eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:02:12.765481 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9470510c-5498-437a-9909-f74e35a724eb admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:12.765762 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9470510c-5498-437a-9909-f74e35a724eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:12.766042 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9470510c-5498-437a-9909-f74e35a724eb admin admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:12.766853 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9470510c-5498-437a-9909-f74e35a724eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:02:12.768470 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9470510c-5498-437a-9909-f74e35a724eb admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:02:12.780989 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1008/2041] 10.222.0.46 () {58 vars in 1121 bytes} [Thu Jun 23 16:02:12 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 16:02:12.785184 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfdd251-f0d5-40c1-bfe5-462e235b917e admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:02:12.796240 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfdd251-f0d5-40c1-bfe5-462e235b917e admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:02:12.801439 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3eb56bb-1f10-4e25-a736-6291f0facc52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:02:12.802112 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3eb56bb-1f10-4e25-a736-6291f0facc52 admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:12.802392 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3eb56bb-1f10-4e25-a736-6291f0facc52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:12.802669 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3eb56bb-1f10-4e25-a736-6291f0facc52 admin admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:12.803429 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3eb56bb-1f10-4e25-a736-6291f0facc52 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:02:12.805090 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3eb56bb-1f10-4e25-a736-6291f0facc52 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:02:12.817185 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1034/2042] 10.222.0.46 () {58 vars in 1121 bytes} [Thu Jun 23 16:02:12 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 16:02:12.822299 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9470510c-5498-437a-9909-f74e35a724eb admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:02:12.832626 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9470510c-5498-437a-9909-f74e35a724eb admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:02:12.838843 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce36b7f-2642-4515-8507-7ea202a1bc1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:02:12.839783 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce36b7f-2642-4515-8507-7ea202a1bc1c admin admin] REQUEST_METHOD: `PUT` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:12.840110 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce36b7f-2642-4515-8507-7ea202a1bc1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:12.840397 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce36b7f-2642-4515-8507-7ea202a1bc1c admin admin] PATH_INFO: `/v3/projects/c8109f8a8e594638b22e14e49b3c7f22/users/388331b0e5d844a494089ff580c96512/roles/e2a5bbf3200a4d4091d72bc7ff78c34d` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:12.856217 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce36b7f-2642-4515-8507-7ea202a1bc1c admin admin] RBAC: Authorizing `identity:create_grant(project_id=c8109f8a8e594638b22e14e49b3c7f22, user_id=388331b0e5d844a494089ff580c96512, role_id=e2a5bbf3200a4d4091d72bc7ff78c34d)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:02:12.858939 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce36b7f-2642-4515-8507-7ea202a1bc1c admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:02:12.876027 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1009/2043] 10.222.0.46 () {60 vars in 1482 bytes} [Thu Jun 23 16:02:12 2022] PUT /identity/v3/projects/c8109f8a8e594638b22e14e49b3c7f22/users/388331b0e5d844a494089ff580c96512/roles/e2a5bbf3200a4d4091d72bc7ff78c34d => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 16:02:12.886760 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79056a3-3caf-43ed-9be2-593441ee9874 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:12.887107 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79056a3-3caf-43ed-9be2-593441ee9874 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:12.887342 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79056a3-3caf-43ed-9be2-593441ee9874 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:12.917231 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-a79056a3-3caf-43ed-9be2-593441ee9874 None None] MFA Rules not processed for user `388331b0e5d844a494089ff580c96512`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:02:12.918491 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-a79056a3-3caf-43ed-9be2-593441ee9874 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:02:12.934744 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:02:13.023033 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1035/2044] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:02:12 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 16:02:13.028464 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce36b7f-2642-4515-8507-7ea202a1bc1c admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:02:13.112686 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce36b7f-2642-4515-8507-7ea202a1bc1c admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:02:13.119040 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ce4b5b-33a1-4c5a-bc11-eaa7913f02d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:02:13.119998 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ce4b5b-33a1-4c5a-bc11-eaa7913f02d7 admin admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:13.120281 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ce4b5b-33a1-4c5a-bc11-eaa7913f02d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:13.120561 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ce4b5b-33a1-4c5a-bc11-eaa7913f02d7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:13.121284 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ce4b5b-33a1-4c5a-bc11-eaa7913f02d7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:02:13.123365 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ce4b5b-33a1-4c5a-bc11-eaa7913f02d7 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:02:13.158800 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1010/2045] 10.222.0.46 () {60 vars in 1152 bytes} [Thu Jun 23 16:02:13 2022] POST /identity/v3/projects => generated 387 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 16:02:13.164091 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79056a3-3caf-43ed-9be2-593441ee9874 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:02:13.177912 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79056a3-3caf-43ed-9be2-593441ee9874 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:02:13.238047 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f796079-1d57-49c3-aa00-1d8680425194 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:02:13.239006 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f796079-1d57-49c3-aa00-1d8680425194 admin admin] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:13.239359 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f796079-1d57-49c3-aa00-1d8680425194 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:13.239788 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f796079-1d57-49c3-aa00-1d8680425194 admin admin] PATH_INFO: `/v3/users` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:13.240534 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f796079-1d57-49c3-aa00-1d8680425194 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:02:13.243205 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f796079-1d57-49c3-aa00-1d8680425194 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:02:13.267234 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1036/2046] 10.222.0.46 () {60 vars in 1143 bytes} [Thu Jun 23 16:02:13 2022] POST /identity/v3/users => generated 300 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 16:02:13.271067 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ce4b5b-33a1-4c5a-bc11-eaa7913f02d7 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:02:13.281826 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ce4b5b-33a1-4c5a-bc11-eaa7913f02d7 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:02:13.286486 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ac8a34-9b19-425c-9262-742a0cf392a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:02:13.287193 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ac8a34-9b19-425c-9262-742a0cf392a1 admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:13.287482 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ac8a34-9b19-425c-9262-742a0cf392a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:13.287768 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ac8a34-9b19-425c-9262-742a0cf392a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:13.288507 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ac8a34-9b19-425c-9262-742a0cf392a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:02:13.290165 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ac8a34-9b19-425c-9262-742a0cf392a1 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:02:13.307251 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1011/2047] 10.222.0.46 () {58 vars in 1121 bytes} [Thu Jun 23 16:02:13 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 16:02:13.308288 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f796079-1d57-49c3-aa00-1d8680425194 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:02:13.320869 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f796079-1d57-49c3-aa00-1d8680425194 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:02:13.329735 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dac27a-7938-41ab-8044-12e35b7f9e83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:02:13.330654 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dac27a-7938-41ab-8044-12e35b7f9e83 admin admin] REQUEST_METHOD: `PUT` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:13.330995 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dac27a-7938-41ab-8044-12e35b7f9e83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:13.331278 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dac27a-7938-41ab-8044-12e35b7f9e83 admin admin] PATH_INFO: `/v3/projects/014b0516201140cc98a26059e1d27247/users/9dfa8233a3104750bf659d2092febb4d/roles/812d3c567c52469cb2b35709fbdc857c` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:13.347179 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8dac27a-7938-41ab-8044-12e35b7f9e83 admin admin] RBAC: Authorizing `identity:create_grant(project_id=014b0516201140cc98a26059e1d27247, user_id=9dfa8233a3104750bf659d2092febb4d, role_id=812d3c567c52469cb2b35709fbdc857c)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:02:13.350023 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8dac27a-7938-41ab-8044-12e35b7f9e83 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:02:13.364689 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1037/2048] 10.222.0.46 () {60 vars in 1482 bytes} [Thu Jun 23 16:02:13 2022] PUT /identity/v3/projects/014b0516201140cc98a26059e1d27247/users/9dfa8233a3104750bf659d2092febb4d/roles/812d3c567c52469cb2b35709fbdc857c => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 16:02:13.369493 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ac8a34-9b19-425c-9262-742a0cf392a1 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:02:13.380411 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ac8a34-9b19-425c-9262-742a0cf392a1 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:02:13.420578 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:02:13.449082 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b10731-96a0-4706-b6d9-9a777e6491f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:02:13.449812 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b10731-96a0-4706-b6d9-9a777e6491f7 admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:13.450125 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b10731-96a0-4706-b6d9-9a777e6491f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:13.450380 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b10731-96a0-4706-b6d9-9a777e6491f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:13.451150 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b10731-96a0-4706-b6d9-9a777e6491f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:02:13.452758 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b10731-96a0-4706-b6d9-9a777e6491f7 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:02:13.466559 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1012/2049] 10.222.0.46 () {58 vars in 1121 bytes} [Thu Jun 23 16:02:13 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 16:02:13.467941 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dac27a-7938-41ab-8044-12e35b7f9e83 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:02:13.479300 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dac27a-7938-41ab-8044-12e35b7f9e83 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:02:13.484450 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3347c8-15aa-4371-9003-e2c7b698b987 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:02:13.485384 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3347c8-15aa-4371-9003-e2c7b698b987 admin admin] REQUEST_METHOD: `PUT` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:13.485672 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3347c8-15aa-4371-9003-e2c7b698b987 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:13.485955 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3347c8-15aa-4371-9003-e2c7b698b987 admin admin] PATH_INFO: `/v3/projects/014b0516201140cc98a26059e1d27247/users/9dfa8233a3104750bf659d2092febb4d/roles/812d3c567c52469cb2b35709fbdc857c` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:13.488033 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3347c8-15aa-4371-9003-e2c7b698b987 admin admin] RBAC: Authorizing `identity:create_grant(project_id=014b0516201140cc98a26059e1d27247, user_id=9dfa8233a3104750bf659d2092febb4d, role_id=812d3c567c52469cb2b35709fbdc857c)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:02:13.490875 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3347c8-15aa-4371-9003-e2c7b698b987 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:02:13.502210 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1038/2050] 10.222.0.46 () {60 vars in 1482 bytes} [Thu Jun 23 16:02:13 2022] PUT /identity/v3/projects/014b0516201140cc98a26059e1d27247/users/9dfa8233a3104750bf659d2092febb4d/roles/812d3c567c52469cb2b35709fbdc857c => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 16:02:13.512731 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435d1f3e-034c-474b-8a86-8b78ce5c5d72 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:13.513030 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435d1f3e-034c-474b-8a86-8b78ce5c5d72 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:13.513316 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435d1f3e-034c-474b-8a86-8b78ce5c5d72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:13.543652 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-435d1f3e-034c-474b-8a86-8b78ce5c5d72 None None] MFA Rules not processed for user `9dfa8233a3104750bf659d2092febb4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:02:13.544913 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-435d1f3e-034c-474b-8a86-8b78ce5c5d72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:02:13.627765 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1013/2051] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:02:13 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 16:02:13.639126 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3347c8-15aa-4371-9003-e2c7b698b987 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:02:13.641099 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-5c3347c8-15aa-4371-9003-e2c7b698b987 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:02:13.723748 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3347c8-15aa-4371-9003-e2c7b698b987 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:02:13.728255 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83c2b84-2e2a-4b8f-a3ca-11805f22ae47 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce18eaaba3a6477684fff1fd98ec8b51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:02:13.729003 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83c2b84-2e2a-4b8f-a3ca-11805f22ae47 service cinder] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:13.729365 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83c2b84-2e2a-4b8f-a3ca-11805f22ae47 service cinder] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:13.729894 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83c2b84-2e2a-4b8f-a3ca-11805f22ae47 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:13.741468 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83c2b84-2e2a-4b8f-a3ca-11805f22ae47 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:02:13.743754 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83c2b84-2e2a-4b8f-a3ca-11805f22ae47 service cinder] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:02:13.755749 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1039/2052] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jun 23 16:02:13 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 16:02:13.785050 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435d1f3e-034c-474b-8a86-8b78ce5c5d72 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:02:13.887198 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435d1f3e-034c-474b-8a86-8b78ce5c5d72 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:02:13.892132 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4d8666-d760-436c-be4c-08ea84a05480 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:02:13.893056 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4d8666-d760-436c-be4c-08ea84a05480 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:13.893353 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4d8666-d760-436c-be4c-08ea84a05480 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:13.893636 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4d8666-d760-436c-be4c-08ea84a05480 admin admin] PATH_INFO: `/v3/users/388331b0e5d844a494089ff580c96512` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:13.894971 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4d8666-d760-436c-be4c-08ea84a05480 admin admin] RBAC: Authorizing `identity:delete_user(user_id=388331b0e5d844a494089ff580c96512)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:02:13.897020 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4d8666-d760-436c-be4c-08ea84a05480 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:02:13.959299 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-da4d8666-d760-436c-be4c-08ea84a05480 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '388331b0e5d844a494089ff580c96512'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:13.959979 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-da4d8666-d760-436c-be4c-08ea84a05480 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '388331b0e5d844a494089ff580c96512'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:13.969657 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-da4d8666-d760-436c-be4c-08ea84a05480 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '388331b0e5d844a494089ff580c96512'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:13.978729 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:02:13.985837 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-da4d8666-d760-436c-be4c-08ea84a05480 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '388331b0e5d844a494089ff580c96512'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:13.986663 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-da4d8666-d760-436c-be4c-08ea84a05480 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '388331b0e5d844a494089ff580c96512'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:14.011800 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1014/2053] 10.222.0.46 () {58 vars in 1223 bytes} [Thu Jun 23 16:02:13 2022] DELETE /identity/v3/users/388331b0e5d844a494089ff580c96512 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 16:02:14.015682 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6973e14c-48e2-4341-97a0-9ded510bae40 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:14.016041 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6973e14c-48e2-4341-97a0-9ded510bae40 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:14.016385 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6973e14c-48e2-4341-97a0-9ded510bae40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:14.037907 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-6973e14c-48e2-4341-97a0-9ded510bae40 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:02:14.039854 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-6973e14c-48e2-4341-97a0-9ded510bae40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:02:14.045539 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:02:14.123348 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1040/2054] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:02:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 16:02:14.133199 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4d8666-d760-436c-be4c-08ea84a05480 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:02:14.134641 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-da4d8666-d760-436c-be4c-08ea84a05480 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:02:14.219837 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4d8666-d760-436c-be4c-08ea84a05480 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:02:14.223784 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7914fc-b836-4d8e-ad08-5aa8bb1343ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51784064951a4619ae09b9167fb0fd0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:02:14.224452 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7914fc-b836-4d8e-ad08-5aa8bb1343ef service neutron] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:14.224733 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7914fc-b836-4d8e-ad08-5aa8bb1343ef service neutron] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:14.225006 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7914fc-b836-4d8e-ad08-5aa8bb1343ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:14.236311 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7914fc-b836-4d8e-ad08-5aa8bb1343ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:02:14.237877 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7914fc-b836-4d8e-ad08-5aa8bb1343ef service neutron] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:02:14.248706 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1015/2055] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jun 23 16:02:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 16:02:14.559214 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6973e14c-48e2-4341-97a0-9ded510bae40 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:02:14.562539 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-6973e14c-48e2-4341-97a0-9ded510bae40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:02:14.582176 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6973e14c-48e2-4341-97a0-9ded510bae40 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:02:14.588713 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245af409-5de2-4e70-bd3e-a4b404896e75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:02:14.590173 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245af409-5de2-4e70-bd3e-a4b404896e75 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:14.590462 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245af409-5de2-4e70-bd3e-a4b404896e75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:14.590965 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245af409-5de2-4e70-bd3e-a4b404896e75 admin admin] PATH_INFO: `/v3/projects/c8109f8a8e594638b22e14e49b3c7f22` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:14.592496 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245af409-5de2-4e70-bd3e-a4b404896e75 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c8109f8a8e594638b22e14e49b3c7f22)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:02:14.595895 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245af409-5de2-4e70-bd3e-a4b404896e75 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:02:14.660497 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-245af409-5de2-4e70-bd3e-a4b404896e75 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c8109f8a8e594638b22e14e49b3c7f22', 'request_id': 'req-245af409-5de2-4e70-bd3e-a4b404896e75'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:14.667033 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-245af409-5de2-4e70-bd3e-a4b404896e75 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c8109f8a8e594638b22e14e49b3c7f22', 'request_id': 'req-245af409-5de2-4e70-bd3e-a4b404896e75'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:14.673778 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-245af409-5de2-4e70-bd3e-a4b404896e75 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c8109f8a8e594638b22e14e49b3c7f22', 'request_id': 'req-245af409-5de2-4e70-bd3e-a4b404896e75'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:14.691018 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-245af409-5de2-4e70-bd3e-a4b404896e75 admin admin] The token cache is being invalidate because project c8109f8a8e594638b22e14e49b3c7f22 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80153) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 16:02:14.691335 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-245af409-5de2-4e70-bd3e-a4b404896e75 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:14.692613 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-245af409-5de2-4e70-bd3e-a4b404896e75 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:14.694607 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1041/2056] 10.222.0.46 () {58 vars in 1232 bytes} [Thu Jun 23 16:02:14 2022] DELETE /identity/v3/projects/c8109f8a8e594638b22e14e49b3c7f22 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 16:02:14.698856 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7914fc-b836-4d8e-ad08-5aa8bb1343ef service neutron] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:02:14.709962 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7914fc-b836-4d8e-ad08-5aa8bb1343ef service neutron] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:02:14.771234 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a70ddc-f155-4005-bbbd-269972b15854 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:02:14.772547 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a70ddc-f155-4005-bbbd-269972b15854 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:14.772983 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a70ddc-f155-4005-bbbd-269972b15854 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:14.773611 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a70ddc-f155-4005-bbbd-269972b15854 admin admin] PATH_INFO: `/v3/users/9dfa8233a3104750bf659d2092febb4d` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:14.775624 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a70ddc-f155-4005-bbbd-269972b15854 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9dfa8233a3104750bf659d2092febb4d)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:02:14.777999 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a70ddc-f155-4005-bbbd-269972b15854 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:02:14.833701 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-b3a70ddc-f155-4005-bbbd-269972b15854 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9dfa8233a3104750bf659d2092febb4d'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:14.838141 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-b3a70ddc-f155-4005-bbbd-269972b15854 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9dfa8233a3104750bf659d2092febb4d'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:14.847318 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-b3a70ddc-f155-4005-bbbd-269972b15854 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9dfa8233a3104750bf659d2092febb4d'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:14.856561 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-b3a70ddc-f155-4005-bbbd-269972b15854 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9dfa8233a3104750bf659d2092febb4d'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:14.857394 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-b3a70ddc-f155-4005-bbbd-269972b15854 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9dfa8233a3104750bf659d2092febb4d'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:14.873023 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:02:14.884392 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1016/2057] 10.222.0.46 () {58 vars in 1223 bytes} [Thu Jun 23 16:02:14 2022] DELETE /identity/v3/users/9dfa8233a3104750bf659d2092febb4d => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 16:02:15.458445 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245af409-5de2-4e70-bd3e-a4b404896e75 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:02:15.459931 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-245af409-5de2-4e70-bd3e-a4b404896e75 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:02:15.474473 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245af409-5de2-4e70-bd3e-a4b404896e75 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:02:15.542217 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20040bfd-8f8e-4bd4-b195-66be2e047bb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:02:15.543239 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20040bfd-8f8e-4bd4-b195-66be2e047bb7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:15.543566 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20040bfd-8f8e-4bd4-b195-66be2e047bb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:15.543875 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20040bfd-8f8e-4bd4-b195-66be2e047bb7 admin admin] PATH_INFO: `/v3/projects/014b0516201140cc98a26059e1d27247` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:15.545065 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20040bfd-8f8e-4bd4-b195-66be2e047bb7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=014b0516201140cc98a26059e1d27247)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:02:15.547014 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20040bfd-8f8e-4bd4-b195-66be2e047bb7 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:02:15.587225 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:02:15.598096 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-20040bfd-8f8e-4bd4-b195-66be2e047bb7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '014b0516201140cc98a26059e1d27247', 'request_id': 'req-20040bfd-8f8e-4bd4-b195-66be2e047bb7'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:15.604680 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-20040bfd-8f8e-4bd4-b195-66be2e047bb7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '014b0516201140cc98a26059e1d27247', 'request_id': 'req-20040bfd-8f8e-4bd4-b195-66be2e047bb7'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:15.611408 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-20040bfd-8f8e-4bd4-b195-66be2e047bb7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '014b0516201140cc98a26059e1d27247', 'request_id': 'req-20040bfd-8f8e-4bd4-b195-66be2e047bb7'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:15.626089 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-20040bfd-8f8e-4bd4-b195-66be2e047bb7 admin admin] The token cache is being invalidate because project 014b0516201140cc98a26059e1d27247 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80153) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 16:02:15.626387 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-20040bfd-8f8e-4bd4-b195-66be2e047bb7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:15.627052 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-20040bfd-8f8e-4bd4-b195-66be2e047bb7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:15.628620 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1042/2058] 10.222.0.46 () {58 vars in 1232 bytes} [Thu Jun 23 16:02:15 2022] DELETE /identity/v3/projects/014b0516201140cc98a26059e1d27247 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 16:02:15.633702 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f3797c-2e54-47e2-b3b1-30832f5b698b None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:15.633999 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f3797c-2e54-47e2-b3b1-30832f5b698b None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:15.634281 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f3797c-2e54-47e2-b3b1-30832f5b698b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:15.653426 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-f8f3797c-2e54-47e2-b3b1-30832f5b698b None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:02:15.654649 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-f8f3797c-2e54-47e2-b3b1-30832f5b698b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:02:15.714876 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1017/2059] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:02:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 16:02:15.723858 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec811df9-25ef-4586-a0b2-7ba0cdfbe26b None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:15.724145 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec811df9-25ef-4586-a0b2-7ba0cdfbe26b None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:15.724424 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec811df9-25ef-4586-a0b2-7ba0cdfbe26b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:15.743562 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-ec811df9-25ef-4586-a0b2-7ba0cdfbe26b None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:02:15.744768 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-ec811df9-25ef-4586-a0b2-7ba0cdfbe26b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:02:15.753055 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1043/2060] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:02:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 16:02:15.757980 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f3797c-2e54-47e2-b3b1-30832f5b698b None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:02:15.767800 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f3797c-2e54-47e2-b3b1-30832f5b698b None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:02:15.772446 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba20496c-84bc-4545-a784-c544fb7f0148 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:02:15.773691 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba20496c-84bc-4545-a784-c544fb7f0148 admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:15.773983 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba20496c-84bc-4545-a784-c544fb7f0148 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:15.774263 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba20496c-84bc-4545-a784-c544fb7f0148 admin admin] PATH_INFO: `/v3/domains` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:15.775037 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba20496c-84bc-4545-a784-c544fb7f0148 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 16:02:15.775626 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba20496c-84bc-4545-a784-c544fb7f0148 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:02:15.777224 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba20496c-84bc-4545-a784-c544fb7f0148 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:02:15.790173 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1018/2061] 10.222.0.46 () {58 vars in 1152 bytes} [Thu Jun 23 16:02:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 16:02:15.794762 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec811df9-25ef-4586-a0b2-7ba0cdfbe26b None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:02:15.805851 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec811df9-25ef-4586-a0b2-7ba0cdfbe26b None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:02:15.810774 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66fb2e0-6fc1-4edc-8f42-ed2256a7165a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:02:15.811444 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66fb2e0-6fc1-4edc-8f42-ed2256a7165a admin admin] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:15.811754 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66fb2e0-6fc1-4edc-8f42-ed2256a7165a admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:15.812064 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66fb2e0-6fc1-4edc-8f42-ed2256a7165a admin admin] PATH_INFO: `/v3/projects` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:15.813511 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66fb2e0-6fc1-4edc-8f42-ed2256a7165a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:02:15.815455 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66fb2e0-6fc1-4edc-8f42-ed2256a7165a admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:02:15.844335 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1044/2062] 10.222.0.46 () {60 vars in 1152 bytes} [Thu Jun 23 16:02:15 2022] POST /identity/v3/projects => generated 393 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 16:02:15.848518 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba20496c-84bc-4545-a784-c544fb7f0148 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:02:15.859039 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba20496c-84bc-4545-a784-c544fb7f0148 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:02:15.916599 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ef97bc-4a29-4a79-a391-31eefebd616e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:02:15.917286 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ef97bc-4a29-4a79-a391-31eefebd616e admin admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:15.917572 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ef97bc-4a29-4a79-a391-31eefebd616e admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:15.917851 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ef97bc-4a29-4a79-a391-31eefebd616e admin admin] PATH_INFO: `/v3/users` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:15.918577 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ef97bc-4a29-4a79-a391-31eefebd616e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:02:15.920853 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ef97bc-4a29-4a79-a391-31eefebd616e admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:02:15.941236 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1019/2063] 10.222.0.46 () {60 vars in 1143 bytes} [Thu Jun 23 16:02:15 2022] POST /identity/v3/users => generated 297 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 16:02:15.946146 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66fb2e0-6fc1-4edc-8f42-ed2256a7165a admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:02:15.956636 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66fb2e0-6fc1-4edc-8f42-ed2256a7165a admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:02:15.961967 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71a0de5-ed4d-4eac-bc9a-19d98788c17a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:02:15.962623 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71a0de5-ed4d-4eac-bc9a-19d98788c17a admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:15.962950 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71a0de5-ed4d-4eac-bc9a-19d98788c17a admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:15.963236 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71a0de5-ed4d-4eac-bc9a-19d98788c17a admin admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:15.963955 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71a0de5-ed4d-4eac-bc9a-19d98788c17a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:02:15.965563 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71a0de5-ed4d-4eac-bc9a-19d98788c17a admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:02:15.978084 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1045/2064] 10.222.0.46 () {58 vars in 1121 bytes} [Thu Jun 23 16:02:15 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 16:02:15.982105 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ef97bc-4a29-4a79-a391-31eefebd616e admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:02:15.992306 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ef97bc-4a29-4a79-a391-31eefebd616e admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:02:15.997443 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e326d3e2-f04a-4747-aceb-d8c9481078bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:02:15.998117 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e326d3e2-f04a-4747-aceb-d8c9481078bd admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:15.998401 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e326d3e2-f04a-4747-aceb-d8c9481078bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:15.998784 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e326d3e2-f04a-4747-aceb-d8c9481078bd admin admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:15.999532 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e326d3e2-f04a-4747-aceb-d8c9481078bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:02:16.001176 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e326d3e2-f04a-4747-aceb-d8c9481078bd admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:02:16.012819 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1020/2065] 10.222.0.46 () {58 vars in 1121 bytes} [Thu Jun 23 16:02:15 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 16:02:16.017200 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71a0de5-ed4d-4eac-bc9a-19d98788c17a admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:02:16.028717 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71a0de5-ed4d-4eac-bc9a-19d98788c17a admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:02:16.036123 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a0508d-8595-4618-aa9c-eaba0d55128d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:02:16.037030 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a0508d-8595-4618-aa9c-eaba0d55128d admin admin] REQUEST_METHOD: `PUT` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:16.037310 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a0508d-8595-4618-aa9c-eaba0d55128d admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:16.037588 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a0508d-8595-4618-aa9c-eaba0d55128d admin admin] PATH_INFO: `/v3/projects/078e3aa03b55436cbdb838c9c9eda897/users/7baf215827814ddcbfbe7d70df2907ea/roles/e2a5bbf3200a4d4091d72bc7ff78c34d` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:16.062005 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a0508d-8595-4618-aa9c-eaba0d55128d admin admin] RBAC: Authorizing `identity:create_grant(project_id=078e3aa03b55436cbdb838c9c9eda897, user_id=7baf215827814ddcbfbe7d70df2907ea, role_id=e2a5bbf3200a4d4091d72bc7ff78c34d)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:02:16.065054 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a0508d-8595-4618-aa9c-eaba0d55128d admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:02:16.092242 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1046/2066] 10.222.0.46 () {60 vars in 1482 bytes} [Thu Jun 23 16:02:16 2022] PUT /identity/v3/projects/078e3aa03b55436cbdb838c9c9eda897/users/7baf215827814ddcbfbe7d70df2907ea/roles/e2a5bbf3200a4d4091d72bc7ff78c34d => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 16:02:16.094572 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8919bd-df3c-4651-b777-982db21f9ba5 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:16.095232 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8919bd-df3c-4651-b777-982db21f9ba5 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:16.095522 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8919bd-df3c-4651-b777-982db21f9ba5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:16.126637 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-af8919bd-df3c-4651-b777-982db21f9ba5 None None] MFA Rules not processed for user `7baf215827814ddcbfbe7d70df2907ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:02:16.127925 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-af8919bd-df3c-4651-b777-982db21f9ba5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:02:16.236675 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:02:16.249599 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1021/2067] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:02:16 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 16:02:16.254131 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a0508d-8595-4618-aa9c-eaba0d55128d admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:02:16.341396 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a0508d-8595-4618-aa9c-eaba0d55128d admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:02:16.346938 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f128145-03fe-46f7-b3d4-c13cfadc7acd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:02:16.347535 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f128145-03fe-46f7-b3d4-c13cfadc7acd admin admin] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:16.347818 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f128145-03fe-46f7-b3d4-c13cfadc7acd admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:16.348097 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f128145-03fe-46f7-b3d4-c13cfadc7acd admin admin] PATH_INFO: `/v3/projects` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:16.348844 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f128145-03fe-46f7-b3d4-c13cfadc7acd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:02:16.350872 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f128145-03fe-46f7-b3d4-c13cfadc7acd admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:02:16.386035 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1047/2068] 10.222.0.46 () {60 vars in 1152 bytes} [Thu Jun 23 16:02:16 2022] POST /identity/v3/projects => generated 391 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 16:02:16.387664 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8919bd-df3c-4651-b777-982db21f9ba5 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:02:16.401215 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8919bd-df3c-4651-b777-982db21f9ba5 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:02:16.477685 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa8de1f-c49e-4ff3-875a-d21c1729e2f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:02:16.478372 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa8de1f-c49e-4ff3-875a-d21c1729e2f1 admin admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:16.478661 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa8de1f-c49e-4ff3-875a-d21c1729e2f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:16.478986 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa8de1f-c49e-4ff3-875a-d21c1729e2f1 admin admin] PATH_INFO: `/v3/users` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:16.479699 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa8de1f-c49e-4ff3-875a-d21c1729e2f1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:02:16.481624 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa8de1f-c49e-4ff3-875a-d21c1729e2f1 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:02:16.505650 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1022/2069] 10.222.0.46 () {60 vars in 1143 bytes} [Thu Jun 23 16:02:16 2022] POST /identity/v3/users => generated 302 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 16:02:16.510214 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f128145-03fe-46f7-b3d4-c13cfadc7acd admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:02:16.520553 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f128145-03fe-46f7-b3d4-c13cfadc7acd admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:02:16.526197 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12732754-d41a-451f-b42c-38733eb8cd0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:02:16.526915 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12732754-d41a-451f-b42c-38733eb8cd0f admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:16.527200 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12732754-d41a-451f-b42c-38733eb8cd0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:16.527475 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12732754-d41a-451f-b42c-38733eb8cd0f admin admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:16.528207 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12732754-d41a-451f-b42c-38733eb8cd0f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:02:16.529845 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12732754-d41a-451f-b42c-38733eb8cd0f admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:02:16.541307 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1048/2070] 10.222.0.46 () {58 vars in 1121 bytes} [Thu Jun 23 16:02:16 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 16:02:16.545669 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa8de1f-c49e-4ff3-875a-d21c1729e2f1 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:02:16.557398 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa8de1f-c49e-4ff3-875a-d21c1729e2f1 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:02:16.562093 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd9fe2a-c768-489f-bd3e-49efd9f1a0d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:02:16.563035 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd9fe2a-c768-489f-bd3e-49efd9f1a0d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:16.563303 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd9fe2a-c768-489f-bd3e-49efd9f1a0d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:16.563597 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd9fe2a-c768-489f-bd3e-49efd9f1a0d9 admin admin] PATH_INFO: `/v3/projects/f8128fc7a9314726aa205caef9cc952f/users/83d18fadbd3f448fa17b6d4df8eb3eee/roles/812d3c567c52469cb2b35709fbdc857c` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:16.581722 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd9fe2a-c768-489f-bd3e-49efd9f1a0d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f8128fc7a9314726aa205caef9cc952f, user_id=83d18fadbd3f448fa17b6d4df8eb3eee, role_id=812d3c567c52469cb2b35709fbdc857c)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:02:16.584510 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd9fe2a-c768-489f-bd3e-49efd9f1a0d9 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:02:16.599195 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1023/2071] 10.222.0.46 () {60 vars in 1482 bytes} [Thu Jun 23 16:02:16 2022] PUT /identity/v3/projects/f8128fc7a9314726aa205caef9cc952f/users/83d18fadbd3f448fa17b6d4df8eb3eee/roles/812d3c567c52469cb2b35709fbdc857c => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 16:02:16.603971 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12732754-d41a-451f-b42c-38733eb8cd0f admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:02:16.614565 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12732754-d41a-451f-b42c-38733eb8cd0f admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:02:16.647833 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:02:16.674913 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53837946-85bd-473c-95ae-bc70ed4195a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:02:16.675907 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53837946-85bd-473c-95ae-bc70ed4195a0 admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:16.676187 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53837946-85bd-473c-95ae-bc70ed4195a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:16.676459 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53837946-85bd-473c-95ae-bc70ed4195a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:16.677188 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53837946-85bd-473c-95ae-bc70ed4195a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:02:16.678798 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53837946-85bd-473c-95ae-bc70ed4195a0 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:02:16.688505 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1049/2072] 10.222.0.46 () {58 vars in 1121 bytes} [Thu Jun 23 16:02:16 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 16:02:16.693006 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd9fe2a-c768-489f-bd3e-49efd9f1a0d9 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:02:16.704434 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd9fe2a-c768-489f-bd3e-49efd9f1a0d9 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:02:16.709227 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bce6c7c-e1be-4f77-bbe1-eee17280e52a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:02:16.710138 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bce6c7c-e1be-4f77-bbe1-eee17280e52a admin admin] REQUEST_METHOD: `PUT` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:16.710426 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bce6c7c-e1be-4f77-bbe1-eee17280e52a admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:16.710731 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bce6c7c-e1be-4f77-bbe1-eee17280e52a admin admin] PATH_INFO: `/v3/projects/f8128fc7a9314726aa205caef9cc952f/users/83d18fadbd3f448fa17b6d4df8eb3eee/roles/812d3c567c52469cb2b35709fbdc857c` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:16.713307 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bce6c7c-e1be-4f77-bbe1-eee17280e52a admin admin] RBAC: Authorizing `identity:create_grant(project_id=f8128fc7a9314726aa205caef9cc952f, user_id=83d18fadbd3f448fa17b6d4df8eb3eee, role_id=812d3c567c52469cb2b35709fbdc857c)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:02:16.716092 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bce6c7c-e1be-4f77-bbe1-eee17280e52a admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:02:16.727075 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1024/2073] 10.222.0.46 () {60 vars in 1482 bytes} [Thu Jun 23 16:02:16 2022] PUT /identity/v3/projects/f8128fc7a9314726aa205caef9cc952f/users/83d18fadbd3f448fa17b6d4df8eb3eee/roles/812d3c567c52469cb2b35709fbdc857c => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 16:02:16.737267 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf19558-2830-4f21-ba37-018d3aa493b4 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:16.737562 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf19558-2830-4f21-ba37-018d3aa493b4 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:16.737840 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf19558-2830-4f21-ba37-018d3aa493b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:16.766112 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-9cf19558-2830-4f21-ba37-018d3aa493b4 None None] MFA Rules not processed for user `83d18fadbd3f448fa17b6d4df8eb3eee`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:02:16.767399 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-9cf19558-2830-4f21-ba37-018d3aa493b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:02:16.840502 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1050/2074] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:02:16 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 16:02:16.852754 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bce6c7c-e1be-4f77-bbe1-eee17280e52a admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:02:16.854359 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-8bce6c7c-e1be-4f77-bbe1-eee17280e52a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:02:16.914492 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bce6c7c-e1be-4f77-bbe1-eee17280e52a admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:02:16.918581 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b197b13-2789-4ca8-988c-701b598973bc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce18eaaba3a6477684fff1fd98ec8b51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:02:16.919283 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b197b13-2789-4ca8-988c-701b598973bc service cinder] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:16.919592 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b197b13-2789-4ca8-988c-701b598973bc service cinder] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:16.919859 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b197b13-2789-4ca8-988c-701b598973bc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:16.929621 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b197b13-2789-4ca8-988c-701b598973bc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:02:16.931609 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b197b13-2789-4ca8-988c-701b598973bc service cinder] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:02:16.942288 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1025/2075] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jun 23 16:02:16 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 16:02:16.991427 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf19558-2830-4f21-ba37-018d3aa493b4 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:02:17.002149 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf19558-2830-4f21-ba37-018d3aa493b4 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:02:17.006346 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9f223f-68f5-4329-98c3-855a4233ff8f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce18eaaba3a6477684fff1fd98ec8b51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:02:17.007037 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9f223f-68f5-4329-98c3-855a4233ff8f service cinder] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:17.007321 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9f223f-68f5-4329-98c3-855a4233ff8f service cinder] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:17.007597 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9f223f-68f5-4329-98c3-855a4233ff8f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:17.070257 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9f223f-68f5-4329-98c3-855a4233ff8f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:02:17.072515 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9f223f-68f5-4329-98c3-855a4233ff8f service cinder] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:02:17.084128 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1051/2076] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jun 23 16:02:16 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 16:02:26.937487 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=73,keystone:INSERT=5 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:02:27.079608 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=75 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:02:30.256173 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b197b13-2789-4ca8-988c-701b598973bc service cinder] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:02:30.257707 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-3b197b13-2789-4ca8-988c-701b598973bc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:02:30.334263 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b197b13-2789-4ca8-988c-701b598973bc service cinder] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:02:30.342846 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ef6926-c15f-4fa1-a420-5750aa91dd56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:02:30.343816 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ef6926-c15f-4fa1-a420-5750aa91dd56 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:30.344206 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ef6926-c15f-4fa1-a420-5750aa91dd56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:30.344484 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ef6926-c15f-4fa1-a420-5750aa91dd56 admin admin] PATH_INFO: `/v3/users/c4d45077e6814b0e9b1391d98fec2678` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:30.345813 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ef6926-c15f-4fa1-a420-5750aa91dd56 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c4d45077e6814b0e9b1391d98fec2678)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:02:30.348088 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ef6926-c15f-4fa1-a420-5750aa91dd56 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:02:30.440081 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-97ef6926-c15f-4fa1-a420-5750aa91dd56 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c4d45077e6814b0e9b1391d98fec2678'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:30.441132 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-97ef6926-c15f-4fa1-a420-5750aa91dd56 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c4d45077e6814b0e9b1391d98fec2678'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:30.458106 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-97ef6926-c15f-4fa1-a420-5750aa91dd56 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c4d45077e6814b0e9b1391d98fec2678'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:30.461667 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-97ef6926-c15f-4fa1-a420-5750aa91dd56 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c4d45077e6814b0e9b1391d98fec2678'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:30.462450 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-97ef6926-c15f-4fa1-a420-5750aa91dd56 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c4d45077e6814b0e9b1391d98fec2678'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:30.487834 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1026/2077] 10.222.0.46 () {58 vars in 1223 bytes} [Thu Jun 23 16:02:30 2022] DELETE /identity/v3/users/c4d45077e6814b0e9b1391d98fec2678 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 16:02:30.496646 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64e006b-c1bd-4e9b-b8e9-5e4181d5b11d None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:30.497376 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64e006b-c1bd-4e9b-b8e9-5e4181d5b11d None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:30.497816 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64e006b-c1bd-4e9b-b8e9-5e4181d5b11d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:30.519409 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-e64e006b-c1bd-4e9b-b8e9-5e4181d5b11d None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:02:30.520621 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-e64e006b-c1bd-4e9b-b8e9-5e4181d5b11d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:02:30.544290 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ef6926-c15f-4fa1-a420-5750aa91dd56 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:02:30.545861 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-97ef6926-c15f-4fa1-a420-5750aa91dd56 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:02:30.618358 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1052/2078] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:02:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 16:02:30.627306 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64e006b-c1bd-4e9b-b8e9-5e4181d5b11d None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:02:30.628933 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-e64e006b-c1bd-4e9b-b8e9-5e4181d5b11d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:02:30.666553 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ef6926-c15f-4fa1-a420-5750aa91dd56 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:02:30.673557 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4194d7-5f90-42b4-a578-ef14de6622cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:02:30.674885 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4194d7-5f90-42b4-a578-ef14de6622cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:30.675486 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4194d7-5f90-42b4-a578-ef14de6622cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:30.675839 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4194d7-5f90-42b4-a578-ef14de6622cd admin admin] PATH_INFO: `/v3/users/7baf215827814ddcbfbe7d70df2907ea` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:30.677179 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4194d7-5f90-42b4-a578-ef14de6622cd admin admin] RBAC: Authorizing `identity:delete_user(user_id=7baf215827814ddcbfbe7d70df2907ea)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:02:30.680563 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4194d7-5f90-42b4-a578-ef14de6622cd admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:02:30.716861 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:02:30.744557 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64e006b-c1bd-4e9b-b8e9-5e4181d5b11d None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:02:30.750768 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a993a33-b17f-420c-b8f3-32b09362a7ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51784064951a4619ae09b9167fb0fd0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:02:30.751597 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a993a33-b17f-420c-b8f3-32b09362a7ad service neutron] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:30.751895 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a993a33-b17f-420c-b8f3-32b09362a7ad service neutron] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:30.752171 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a993a33-b17f-420c-b8f3-32b09362a7ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:30.762619 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-6b4194d7-5f90-42b4-a578-ef14de6622cd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7baf215827814ddcbfbe7d70df2907ea'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:30.764811 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-6b4194d7-5f90-42b4-a578-ef14de6622cd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7baf215827814ddcbfbe7d70df2907ea'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:30.769801 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a993a33-b17f-420c-b8f3-32b09362a7ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:02:30.771602 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a993a33-b17f-420c-b8f3-32b09362a7ad service neutron] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:02:30.777630 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-6b4194d7-5f90-42b4-a578-ef14de6622cd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7baf215827814ddcbfbe7d70df2907ea'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:30.795384 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-6b4194d7-5f90-42b4-a578-ef14de6622cd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7baf215827814ddcbfbe7d70df2907ea'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:30.796195 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-6b4194d7-5f90-42b4-a578-ef14de6622cd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7baf215827814ddcbfbe7d70df2907ea'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:30.816164 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1053/2079] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jun 23 16:02:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 16:02:30.829081 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1027/2080] 10.222.0.46 () {58 vars in 1223 bytes} [Thu Jun 23 16:02:30 2022] DELETE /identity/v3/users/7baf215827814ddcbfbe7d70df2907ea => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 16:02:30.832340 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1431b5-8e42-4f8e-8eed-f1a242191c06 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:30.832639 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1431b5-8e42-4f8e-8eed-f1a242191c06 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:30.833852 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1431b5-8e42-4f8e-8eed-f1a242191c06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:30.875387 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-3e1431b5-8e42-4f8e-8eed-f1a242191c06 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:02:30.877534 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-3e1431b5-8e42-4f8e-8eed-f1a242191c06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:02:30.982736 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1054/2081] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:02:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 16:02:30.996294 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4194d7-5f90-42b4-a578-ef14de6622cd admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:02:31.000246 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-6b4194d7-5f90-42b4-a578-ef14de6622cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:02:31.097887 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4194d7-5f90-42b4-a578-ef14de6622cd admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:02:31.103878 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc29be3-fc29-4b00-8b97-ac9e6adef9e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51784064951a4619ae09b9167fb0fd0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:02:31.104784 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc29be3-fc29-4b00-8b97-ac9e6adef9e5 service neutron] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:31.105075 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc29be3-fc29-4b00-8b97-ac9e6adef9e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:31.105971 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc29be3-fc29-4b00-8b97-ac9e6adef9e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:31.123207 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc29be3-fc29-4b00-8b97-ac9e6adef9e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:02:31.124696 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc29be3-fc29-4b00-8b97-ac9e6adef9e5 service neutron] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:02:31.141992 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1028/2082] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jun 23 16:02:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 16:02:31.496825 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1431b5-8e42-4f8e-8eed-f1a242191c06 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:02:31.499888 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-3e1431b5-8e42-4f8e-8eed-f1a242191c06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:02:31.514737 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1431b5-8e42-4f8e-8eed-f1a242191c06 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:02:31.520698 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc29be3-fc29-4b00-8b97-ac9e6adef9e5 service neutron] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:02:31.522645 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-1bc29be3-fc29-4b00-8b97-ac9e6adef9e5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:02:31.529837 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b4d113-edd0-45cb-b16e-55b89fe8b020 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:02:31.546596 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc29be3-fc29-4b00-8b97-ac9e6adef9e5 service neutron] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:02:31.550133 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b4d113-edd0-45cb-b16e-55b89fe8b020 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:31.551743 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b4d113-edd0-45cb-b16e-55b89fe8b020 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:31.560100 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b4d113-edd0-45cb-b16e-55b89fe8b020 admin admin] PATH_INFO: `/v3/projects/078e3aa03b55436cbdb838c9c9eda897` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:31.561762 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aaa72b7-2dc4-4be1-801d-911e1ed09ec0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:02:31.563233 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aaa72b7-2dc4-4be1-801d-911e1ed09ec0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:31.563988 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aaa72b7-2dc4-4be1-801d-911e1ed09ec0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:31.564422 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aaa72b7-2dc4-4be1-801d-911e1ed09ec0 admin admin] PATH_INFO: `/v3/projects/8387833e3174458a970ca46eedcf0918` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:31.565208 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b4d113-edd0-45cb-b16e-55b89fe8b020 admin admin] RBAC: Authorizing `identity:delete_project(project_id=078e3aa03b55436cbdb838c9c9eda897)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:02:31.567090 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aaa72b7-2dc4-4be1-801d-911e1ed09ec0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8387833e3174458a970ca46eedcf0918)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:02:31.569140 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b4d113-edd0-45cb-b16e-55b89fe8b020 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:02:31.573081 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aaa72b7-2dc4-4be1-801d-911e1ed09ec0 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:02:31.650986 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:02:31.668485 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-0aaa72b7-2dc4-4be1-801d-911e1ed09ec0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8387833e3174458a970ca46eedcf0918', 'request_id': 'req-0aaa72b7-2dc4-4be1-801d-911e1ed09ec0'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:31.669438 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-49b4d113-edd0-45cb-b16e-55b89fe8b020 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '078e3aa03b55436cbdb838c9c9eda897', 'request_id': 'req-49b4d113-edd0-45cb-b16e-55b89fe8b020'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:31.675479 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-0aaa72b7-2dc4-4be1-801d-911e1ed09ec0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8387833e3174458a970ca46eedcf0918', 'request_id': 'req-0aaa72b7-2dc4-4be1-801d-911e1ed09ec0'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:31.677740 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-49b4d113-edd0-45cb-b16e-55b89fe8b020 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '078e3aa03b55436cbdb838c9c9eda897', 'request_id': 'req-49b4d113-edd0-45cb-b16e-55b89fe8b020'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:31.684824 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-0aaa72b7-2dc4-4be1-801d-911e1ed09ec0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8387833e3174458a970ca46eedcf0918', 'request_id': 'req-0aaa72b7-2dc4-4be1-801d-911e1ed09ec0'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:31.686751 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-49b4d113-edd0-45cb-b16e-55b89fe8b020 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '078e3aa03b55436cbdb838c9c9eda897', 'request_id': 'req-49b4d113-edd0-45cb-b16e-55b89fe8b020'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:31.704887 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-0aaa72b7-2dc4-4be1-801d-911e1ed09ec0 admin admin] The token cache is being invalidate because project 8387833e3174458a970ca46eedcf0918 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80154) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 16:02:31.705442 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-0aaa72b7-2dc4-4be1-801d-911e1ed09ec0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:31.706599 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-0aaa72b7-2dc4-4be1-801d-911e1ed09ec0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:31.709624 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1029/2083] 10.222.0.46 () {58 vars in 1232 bytes} [Thu Jun 23 16:02:31 2022] DELETE /identity/v3/projects/8387833e3174458a970ca46eedcf0918 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 16:02:31.712473 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-49b4d113-edd0-45cb-b16e-55b89fe8b020 admin admin] The token cache is being invalidate because project 078e3aa03b55436cbdb838c9c9eda897 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80153) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 16:02:31.713171 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-49b4d113-edd0-45cb-b16e-55b89fe8b020 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:31.713976 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-49b4d113-edd0-45cb-b16e-55b89fe8b020 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:31.715912 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1055/2084] 10.222.0.46 () {58 vars in 1232 bytes} [Thu Jun 23 16:02:31 2022] DELETE /identity/v3/projects/078e3aa03b55436cbdb838c9c9eda897 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 16:02:31.716277 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aaa72b7-2dc4-4be1-801d-911e1ed09ec0 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:02:31.718603 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-0aaa72b7-2dc4-4be1-801d-911e1ed09ec0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:02:31.720019 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b4d113-edd0-45cb-b16e-55b89fe8b020 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:02:31.721499 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-49b4d113-edd0-45cb-b16e-55b89fe8b020 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:02:31.753096 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:02:31.792707 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b4d113-edd0-45cb-b16e-55b89fe8b020 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:02:31.797399 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aaa72b7-2dc4-4be1-801d-911e1ed09ec0 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:02:31.797527 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af22237d-441a-42c7-a058-ea0066cb1df5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:02:31.798341 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af22237d-441a-42c7-a058-ea0066cb1df5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:31.798646 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af22237d-441a-42c7-a058-ea0066cb1df5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:31.799366 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af22237d-441a-42c7-a058-ea0066cb1df5 admin admin] PATH_INFO: `/v3/users/83d18fadbd3f448fa17b6d4df8eb3eee` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:31.800272 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af22237d-441a-42c7-a058-ea0066cb1df5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=83d18fadbd3f448fa17b6d4df8eb3eee)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:02:31.802200 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af22237d-441a-42c7-a058-ea0066cb1df5 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:02:31.803791 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5798cea8-dfc3-4bad-98d5-d8c2b09b1e62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:02:31.804595 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5798cea8-dfc3-4bad-98d5-d8c2b09b1e62 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:31.804909 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5798cea8-dfc3-4bad-98d5-d8c2b09b1e62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:31.805219 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5798cea8-dfc3-4bad-98d5-d8c2b09b1e62 admin admin] PATH_INFO: `/v3/users/c7e1fe2c351a446db003d57630f3357b` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:31.806461 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5798cea8-dfc3-4bad-98d5-d8c2b09b1e62 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c7e1fe2c351a446db003d57630f3357b)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:02:31.808500 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5798cea8-dfc3-4bad-98d5-d8c2b09b1e62 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:02:31.858191 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-af22237d-441a-42c7-a058-ea0066cb1df5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '83d18fadbd3f448fa17b6d4df8eb3eee'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:31.861060 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-5798cea8-dfc3-4bad-98d5-d8c2b09b1e62 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c7e1fe2c351a446db003d57630f3357b'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:31.861761 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-5798cea8-dfc3-4bad-98d5-d8c2b09b1e62 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c7e1fe2c351a446db003d57630f3357b'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:31.868441 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-af22237d-441a-42c7-a058-ea0066cb1df5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '83d18fadbd3f448fa17b6d4df8eb3eee'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:31.869184 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-af22237d-441a-42c7-a058-ea0066cb1df5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '83d18fadbd3f448fa17b6d4df8eb3eee'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:31.870001 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-5798cea8-dfc3-4bad-98d5-d8c2b09b1e62 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c7e1fe2c351a446db003d57630f3357b'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:31.870127 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-af22237d-441a-42c7-a058-ea0066cb1df5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '83d18fadbd3f448fa17b6d4df8eb3eee'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:31.880620 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-5798cea8-dfc3-4bad-98d5-d8c2b09b1e62 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c7e1fe2c351a446db003d57630f3357b'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:31.881281 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-5798cea8-dfc3-4bad-98d5-d8c2b09b1e62 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c7e1fe2c351a446db003d57630f3357b'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:31.886769 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-af22237d-441a-42c7-a058-ea0066cb1df5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '83d18fadbd3f448fa17b6d4df8eb3eee'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:31.908650 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1056/2085] 10.222.0.46 () {58 vars in 1223 bytes} [Thu Jun 23 16:02:31 2022] DELETE /identity/v3/users/83d18fadbd3f448fa17b6d4df8eb3eee => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 16:02:31.909086 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1030/2086] 10.222.0.46 () {58 vars in 1223 bytes} [Thu Jun 23 16:02:31 2022] DELETE /identity/v3/users/c7e1fe2c351a446db003d57630f3357b => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 16:02:31.947449 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af22237d-441a-42c7-a058-ea0066cb1df5 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:02:31.948907 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-af22237d-441a-42c7-a058-ea0066cb1df5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:02:32.016201 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:02:32.052453 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af22237d-441a-42c7-a058-ea0066cb1df5 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:02:32.133511 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913765c1-e0d8-4bc6-b0d8-ac6329a3ab71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:02:32.134580 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913765c1-e0d8-4bc6-b0d8-ac6329a3ab71 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:32.134894 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913765c1-e0d8-4bc6-b0d8-ac6329a3ab71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:32.135173 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913765c1-e0d8-4bc6-b0d8-ac6329a3ab71 admin admin] PATH_INFO: `/v3/users/329682080b7a40b18cf0d24bbf405934` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:32.136432 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913765c1-e0d8-4bc6-b0d8-ac6329a3ab71 admin admin] RBAC: Authorizing `identity:delete_user(user_id=329682080b7a40b18cf0d24bbf405934)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:02:32.138384 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913765c1-e0d8-4bc6-b0d8-ac6329a3ab71 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:02:32.194554 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-913765c1-e0d8-4bc6-b0d8-ac6329a3ab71 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '329682080b7a40b18cf0d24bbf405934'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:32.202970 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-913765c1-e0d8-4bc6-b0d8-ac6329a3ab71 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '329682080b7a40b18cf0d24bbf405934'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:32.204055 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-913765c1-e0d8-4bc6-b0d8-ac6329a3ab71 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '329682080b7a40b18cf0d24bbf405934'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:32.204799 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-913765c1-e0d8-4bc6-b0d8-ac6329a3ab71 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '329682080b7a40b18cf0d24bbf405934'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:32.219078 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-913765c1-e0d8-4bc6-b0d8-ac6329a3ab71 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '329682080b7a40b18cf0d24bbf405934'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:32.235821 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1057/2087] 10.222.0.46 () {58 vars in 1223 bytes} [Thu Jun 23 16:02:31 2022] DELETE /identity/v3/users/329682080b7a40b18cf0d24bbf405934 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 16:02:32.240648 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e575e4e-da07-45b7-a96d-5370080bbb55 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:32.241021 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e575e4e-da07-45b7-a96d-5370080bbb55 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:32.241423 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e575e4e-da07-45b7-a96d-5370080bbb55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:32.261449 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-6e575e4e-da07-45b7-a96d-5370080bbb55 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:02:32.262760 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-6e575e4e-da07-45b7-a96d-5370080bbb55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:02:32.359503 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913765c1-e0d8-4bc6-b0d8-ac6329a3ab71 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:02:32.360964 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-913765c1-e0d8-4bc6-b0d8-ac6329a3ab71 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:02:32.377057 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1031/2088] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:02:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 16:02:32.377985 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e575e4e-da07-45b7-a96d-5370080bbb55 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:02:32.388443 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-6e575e4e-da07-45b7-a96d-5370080bbb55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:02:32.407655 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e575e4e-da07-45b7-a96d-5370080bbb55 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:02:32.414509 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98453f62-f608-426c-a4eb-607fbcd00ca9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:02:32.415644 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98453f62-f608-426c-a4eb-607fbcd00ca9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:32.416055 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98453f62-f608-426c-a4eb-607fbcd00ca9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:32.416405 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98453f62-f608-426c-a4eb-607fbcd00ca9 admin admin] PATH_INFO: `/v3/projects/f8128fc7a9314726aa205caef9cc952f` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:32.417925 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98453f62-f608-426c-a4eb-607fbcd00ca9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f8128fc7a9314726aa205caef9cc952f)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:02:32.419906 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98453f62-f608-426c-a4eb-607fbcd00ca9 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:02:32.442975 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913765c1-e0d8-4bc6-b0d8-ac6329a3ab71 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:02:32.448798 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ecdd86-4bf1-404c-9aff-e68d9b9cb127 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:02:32.450026 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ecdd86-4bf1-404c-9aff-e68d9b9cb127 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:32.450362 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ecdd86-4bf1-404c-9aff-e68d9b9cb127 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:32.450656 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ecdd86-4bf1-404c-9aff-e68d9b9cb127 admin admin] PATH_INFO: `/v3/projects/b7b8418309e64dd09fe564bfada4e1f8` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:32.452001 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ecdd86-4bf1-404c-9aff-e68d9b9cb127 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b7b8418309e64dd09fe564bfada4e1f8)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:02:32.454068 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ecdd86-4bf1-404c-9aff-e68d9b9cb127 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:02:32.463479 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:02:32.473453 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:02:32.474006 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-98453f62-f608-426c-a4eb-607fbcd00ca9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f8128fc7a9314726aa205caef9cc952f', 'request_id': 'req-98453f62-f608-426c-a4eb-607fbcd00ca9'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:32.481971 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-98453f62-f608-426c-a4eb-607fbcd00ca9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f8128fc7a9314726aa205caef9cc952f', 'request_id': 'req-98453f62-f608-426c-a4eb-607fbcd00ca9'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:32.493568 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-98453f62-f608-426c-a4eb-607fbcd00ca9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f8128fc7a9314726aa205caef9cc952f', 'request_id': 'req-98453f62-f608-426c-a4eb-607fbcd00ca9'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:32.506831 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-98453f62-f608-426c-a4eb-607fbcd00ca9 admin admin] The token cache is being invalidate because project f8128fc7a9314726aa205caef9cc952f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80154) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 16:02:32.507123 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-98453f62-f608-426c-a4eb-607fbcd00ca9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:32.507809 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-98453f62-f608-426c-a4eb-607fbcd00ca9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:32.509806 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-97ecdd86-4bf1-404c-9aff-e68d9b9cb127 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b7b8418309e64dd09fe564bfada4e1f8', 'request_id': 'req-97ecdd86-4bf1-404c-9aff-e68d9b9cb127'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:32.510343 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1032/2089] 10.222.0.46 () {58 vars in 1232 bytes} [Thu Jun 23 16:02:32 2022] DELETE /identity/v3/projects/f8128fc7a9314726aa205caef9cc952f => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 16:02:32.511056 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98453f62-f608-426c-a4eb-607fbcd00ca9 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:02:32.514564 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-98453f62-f608-426c-a4eb-607fbcd00ca9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:02:32.517246 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-97ecdd86-4bf1-404c-9aff-e68d9b9cb127 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b7b8418309e64dd09fe564bfada4e1f8', 'request_id': 'req-97ecdd86-4bf1-404c-9aff-e68d9b9cb127'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:32.526346 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-97ecdd86-4bf1-404c-9aff-e68d9b9cb127 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b7b8418309e64dd09fe564bfada4e1f8', 'request_id': 'req-97ecdd86-4bf1-404c-9aff-e68d9b9cb127'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:32.541109 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-97ecdd86-4bf1-404c-9aff-e68d9b9cb127 admin admin] The token cache is being invalidate because project b7b8418309e64dd09fe564bfada4e1f8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80153) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 16:02:32.541502 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-97ecdd86-4bf1-404c-9aff-e68d9b9cb127 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:32.542215 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-97ecdd86-4bf1-404c-9aff-e68d9b9cb127 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:32.544102 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1058/2090] 10.222.0.46 () {58 vars in 1232 bytes} [Thu Jun 23 16:02:32 2022] DELETE /identity/v3/projects/b7b8418309e64dd09fe564bfada4e1f8 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 16:02:32.545214 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a439a8f-4973-4b42-827b-6b6ff6903bed None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:32.545506 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a439a8f-4973-4b42-827b-6b6ff6903bed None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:32.545786 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a439a8f-4973-4b42-827b-6b6ff6903bed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:32.564936 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-8a439a8f-4973-4b42-827b-6b6ff6903bed None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:02:32.566159 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-8a439a8f-4973-4b42-827b-6b6ff6903bed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:02:32.575431 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98453f62-f608-426c-a4eb-607fbcd00ca9 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:02:32.626459 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e53e4f9-4f76-4908-a0df-197c0797bfb7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51784064951a4619ae09b9167fb0fd0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:02:32.627172 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e53e4f9-4f76-4908-a0df-197c0797bfb7 service neutron] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:32.627463 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e53e4f9-4f76-4908-a0df-197c0797bfb7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:32.627747 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e53e4f9-4f76-4908-a0df-197c0797bfb7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:32.629632 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-5e53e4f9-4f76-4908-a0df-197c0797bfb7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:02:32.630522 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1059/2091] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:02:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 16:02:32.631932 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443ff27e-0a19-40ca-9cd1-f238d34889e2 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:32.632244 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443ff27e-0a19-40ca-9cd1-f238d34889e2 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:32.632525 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443ff27e-0a19-40ca-9cd1-f238d34889e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:32.646404 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e53e4f9-4f76-4908-a0df-197c0797bfb7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:02:32.647924 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e53e4f9-4f76-4908-a0df-197c0797bfb7 service neutron] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:02:32.652392 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-443ff27e-0a19-40ca-9cd1-f238d34889e2 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:02:32.653526 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-443ff27e-0a19-40ca-9cd1-f238d34889e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:02:32.658087 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1033/2092] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jun 23 16:02:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 16:02:32.659431 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbafb05b-19f7-4d4f-9c27-eefc775aa63e None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:32.659730 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbafb05b-19f7-4d4f-9c27-eefc775aa63e None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:32.659999 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbafb05b-19f7-4d4f-9c27-eefc775aa63e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:32.662452 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1060/2093] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:02:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 16:02:32.670006 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cd4545-ad93-4779-8345-6dc223fb34bb None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:32.670542 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cd4545-ad93-4779-8345-6dc223fb34bb None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:32.670866 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cd4545-ad93-4779-8345-6dc223fb34bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:32.685683 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-bbafb05b-19f7-4d4f-9c27-eefc775aa63e None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:02:32.686936 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-bbafb05b-19f7-4d4f-9c27-eefc775aa63e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:02:32.693991 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-68cd4545-ad93-4779-8345-6dc223fb34bb None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:02:32.695245 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1034/2094] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:02:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 16:02:32.695436 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-68cd4545-ad93-4779-8345-6dc223fb34bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:02:32.699962 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbafb05b-19f7-4d4f-9c27-eefc775aa63e None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:02:32.705845 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1061/2095] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:02:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 16:02:32.709259 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbafb05b-19f7-4d4f-9c27-eefc775aa63e None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:02:32.710461 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cd4545-ad93-4779-8345-6dc223fb34bb None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:02:32.719479 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a9219d-3076-498f-a429-de175df1b45f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:02:32.720310 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a9219d-3076-498f-a429-de175df1b45f admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:32.720694 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a9219d-3076-498f-a429-de175df1b45f admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:32.721128 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a9219d-3076-498f-a429-de175df1b45f admin admin] PATH_INFO: `/v3/domains` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:32.721255 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cd4545-ad93-4779-8345-6dc223fb34bb None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:02:32.723407 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a9219d-3076-498f-a429-de175df1b45f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 16:02:32.723798 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a9219d-3076-498f-a429-de175df1b45f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:02:32.726853 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67daefce-4806-4f83-b723-1a3f41592f9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:02:32.727081 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a9219d-3076-498f-a429-de175df1b45f admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:02:32.727724 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67daefce-4806-4f83-b723-1a3f41592f9c admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:32.728087 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67daefce-4806-4f83-b723-1a3f41592f9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:32.728469 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67daefce-4806-4f83-b723-1a3f41592f9c admin admin] PATH_INFO: `/v3/domains` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:32.729163 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67daefce-4806-4f83-b723-1a3f41592f9c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 16:02:32.729487 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67daefce-4806-4f83-b723-1a3f41592f9c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:02:32.731571 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67daefce-4806-4f83-b723-1a3f41592f9c admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:02:32.747213 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1062/2096] 10.222.0.46 () {58 vars in 1152 bytes} [Thu Jun 23 16:02:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 16:02:32.752005 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67daefce-4806-4f83-b723-1a3f41592f9c admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:02:32.753594 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1035/2097] 10.222.0.46 () {58 vars in 1152 bytes} [Thu Jun 23 16:02:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 16:02:32.759362 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a9219d-3076-498f-a429-de175df1b45f admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:02:32.772466 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67daefce-4806-4f83-b723-1a3f41592f9c admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:02:32.779244 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a9219d-3076-498f-a429-de175df1b45f admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:02:32.780762 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f7d17f-ca53-4121-91a9-2b4d09d68dd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:02:32.781877 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f7d17f-ca53-4121-91a9-2b4d09d68dd4 admin admin] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:32.782844 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f7d17f-ca53-4121-91a9-2b4d09d68dd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:32.783024 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f7d17f-ca53-4121-91a9-2b4d09d68dd4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:32.784423 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f7d17f-ca53-4121-91a9-2b4d09d68dd4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:02:32.788327 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f7d17f-ca53-4121-91a9-2b4d09d68dd4 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:02:32.792783 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf09ec7-0a16-4f88-8a0c-9e252fced987 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:02:32.793626 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf09ec7-0a16-4f88-8a0c-9e252fced987 admin admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:32.793922 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf09ec7-0a16-4f88-8a0c-9e252fced987 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:32.794461 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf09ec7-0a16-4f88-8a0c-9e252fced987 admin admin] PATH_INFO: `/v3/projects` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:32.795223 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf09ec7-0a16-4f88-8a0c-9e252fced987 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:02:32.797641 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf09ec7-0a16-4f88-8a0c-9e252fced987 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:02:32.835280 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1036/2098] 10.222.0.46 () {60 vars in 1152 bytes} [Thu Jun 23 16:02:32 2022] POST /identity/v3/projects => generated 399 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 16:02:32.842115 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf09ec7-0a16-4f88-8a0c-9e252fced987 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:02:32.844525 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1063/2099] 10.222.0.46 () {60 vars in 1152 bytes} [Thu Jun 23 16:02:32 2022] POST /identity/v3/projects => generated 367 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 16:02:32.849586 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f7d17f-ca53-4121-91a9-2b4d09d68dd4 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:02:32.937438 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:02:32.946744 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f7d17f-ca53-4121-91a9-2b4d09d68dd4 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:02:32.955810 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e220b97a-10dc-4117-873d-b8daaf52db5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:02:32.957034 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e220b97a-10dc-4117-873d-b8daaf52db5a admin admin] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:32.957378 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e220b97a-10dc-4117-873d-b8daaf52db5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:32.958014 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e220b97a-10dc-4117-873d-b8daaf52db5a admin admin] PATH_INFO: `/v3/users` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:32.960177 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e220b97a-10dc-4117-873d-b8daaf52db5a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:02:32.964549 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e220b97a-10dc-4117-873d-b8daaf52db5a admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:02:32.979554 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=94,keystone:DELETE=4,keystone:INSERT=2 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:02:32.998921 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1064/2100] 10.222.0.46 () {60 vars in 1143 bytes} [Thu Jun 23 16:02:32 2022] POST /identity/v3/users => generated 284 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 16:02:33.003549 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e220b97a-10dc-4117-873d-b8daaf52db5a admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:02:33.007880 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf09ec7-0a16-4f88-8a0c-9e252fced987 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:02:33.013509 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20d7e67-7b3b-45b7-9ec2-9bf0b8f23a64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:02:33.014210 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20d7e67-7b3b-45b7-9ec2-9bf0b8f23a64 admin admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:33.014500 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20d7e67-7b3b-45b7-9ec2-9bf0b8f23a64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:33.015376 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20d7e67-7b3b-45b7-9ec2-9bf0b8f23a64 admin admin] PATH_INFO: `/v3/users` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:33.015559 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20d7e67-7b3b-45b7-9ec2-9bf0b8f23a64 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:02:33.017507 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20d7e67-7b3b-45b7-9ec2-9bf0b8f23a64 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:02:33.023744 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e220b97a-10dc-4117-873d-b8daaf52db5a admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:02:33.036086 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fce28b-9dcf-4223-a443-31ae0bc16448 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:02:33.037023 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fce28b-9dcf-4223-a443-31ae0bc16448 admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:33.037442 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fce28b-9dcf-4223-a443-31ae0bc16448 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:33.038030 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fce28b-9dcf-4223-a443-31ae0bc16448 admin admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:33.038996 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29fce28b-9dcf-4223-a443-31ae0bc16448 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:02:33.041545 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29fce28b-9dcf-4223-a443-31ae0bc16448 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:02:33.045377 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1037/2101] 10.222.0.46 () {60 vars in 1143 bytes} [Thu Jun 23 16:02:32 2022] POST /identity/v3/users => generated 300 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 16:02:33.049922 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20d7e67-7b3b-45b7-9ec2-9bf0b8f23a64 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:02:33.058060 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1065/2102] 10.222.0.46 () {58 vars in 1121 bytes} [Thu Jun 23 16:02:33 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 16:02:33.062236 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fce28b-9dcf-4223-a443-31ae0bc16448 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:02:33.062392 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20d7e67-7b3b-45b7-9ec2-9bf0b8f23a64 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:02:33.069689 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9839a9-325d-4c64-91b6-50ce23cc8803 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:02:33.070568 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9839a9-325d-4c64-91b6-50ce23cc8803 admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:33.070892 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9839a9-325d-4c64-91b6-50ce23cc8803 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:33.071174 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9839a9-325d-4c64-91b6-50ce23cc8803 admin admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:33.071973 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9839a9-325d-4c64-91b6-50ce23cc8803 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:02:33.074144 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9839a9-325d-4c64-91b6-50ce23cc8803 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:02:33.076615 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fce28b-9dcf-4223-a443-31ae0bc16448 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:02:33.082079 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706bda7b-d146-4d0c-8936-c15e06ce48c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:02:33.083013 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706bda7b-d146-4d0c-8936-c15e06ce48c1 admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:33.083295 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706bda7b-d146-4d0c-8936-c15e06ce48c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:33.083571 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706bda7b-d146-4d0c-8936-c15e06ce48c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:33.086810 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1038/2103] 10.222.0.46 () {58 vars in 1121 bytes} [Thu Jun 23 16:02:33 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 16:02:33.091181 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9839a9-325d-4c64-91b6-50ce23cc8803 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:02:33.094755 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706bda7b-d146-4d0c-8936-c15e06ce48c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:02:33.096908 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706bda7b-d146-4d0c-8936-c15e06ce48c1 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:02:33.105946 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9839a9-325d-4c64-91b6-50ce23cc8803 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:02:33.111053 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da821cd7-9e75-4100-861b-d97a668c7d6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:02:33.111882 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da821cd7-9e75-4100-861b-d97a668c7d6d admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:33.112176 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da821cd7-9e75-4100-861b-d97a668c7d6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:33.112457 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da821cd7-9e75-4100-861b-d97a668c7d6d admin admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:33.113421 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da821cd7-9e75-4100-861b-d97a668c7d6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:02:33.115069 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da821cd7-9e75-4100-861b-d97a668c7d6d admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:02:33.123895 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1066/2104] 10.222.0.46 () {58 vars in 1121 bytes} [Thu Jun 23 16:02:33 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 16:02:33.124675 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706bda7b-d146-4d0c-8936-c15e06ce48c1 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:02:33.127662 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1039/2105] 10.222.0.46 () {58 vars in 1121 bytes} [Thu Jun 23 16:02:33 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 16:02:33.128470 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da821cd7-9e75-4100-861b-d97a668c7d6d admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:02:33.129136 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-706bda7b-d146-4d0c-8936-c15e06ce48c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:02:33.138938 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da821cd7-9e75-4100-861b-d97a668c7d6d admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:02:33.143686 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d649e27-ff91-4d0e-8b2a-6286248c3823 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:02:33.144580 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d649e27-ff91-4d0e-8b2a-6286248c3823 admin admin] REQUEST_METHOD: `PUT` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:33.144864 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d649e27-ff91-4d0e-8b2a-6286248c3823 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:33.145146 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d649e27-ff91-4d0e-8b2a-6286248c3823 admin admin] PATH_INFO: `/v3/projects/4333d946851b4293a126a25934d6c355/users/727f426107294de7a850170b93936e42/roles/e2a5bbf3200a4d4091d72bc7ff78c34d` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:33.146184 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706bda7b-d146-4d0c-8936-c15e06ce48c1 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:02:33.151173 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c65d73-fb60-4de8-9dd3-890605cef49a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:02:33.152029 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c65d73-fb60-4de8-9dd3-890605cef49a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:33.152312 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c65d73-fb60-4de8-9dd3-890605cef49a admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:33.152585 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c65d73-fb60-4de8-9dd3-890605cef49a admin admin] PATH_INFO: `/v3/projects/6ddc880458ff4defa7e2ee9da419f258` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:33.153829 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c65d73-fb60-4de8-9dd3-890605cef49a admin admin] RBAC: Authorizing `identity:delete_project(project_id=6ddc880458ff4defa7e2ee9da419f258)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:02:33.155720 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c65d73-fb60-4de8-9dd3-890605cef49a admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:02:33.160593 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d649e27-ff91-4d0e-8b2a-6286248c3823 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4333d946851b4293a126a25934d6c355, user_id=727f426107294de7a850170b93936e42, role_id=e2a5bbf3200a4d4091d72bc7ff78c34d)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:02:33.163620 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d649e27-ff91-4d0e-8b2a-6286248c3823 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:02:33.177379 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1040/2106] 10.222.0.46 () {60 vars in 1482 bytes} [Thu Jun 23 16:02:33 2022] PUT /identity/v3/projects/4333d946851b4293a126a25934d6c355/users/727f426107294de7a850170b93936e42/roles/e2a5bbf3200a4d4091d72bc7ff78c34d => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 16:02:33.178172 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d649e27-ff91-4d0e-8b2a-6286248c3823 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:02:33.216373 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-82c65d73-fb60-4de8-9dd3-890605cef49a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6ddc880458ff4defa7e2ee9da419f258', 'request_id': 'req-82c65d73-fb60-4de8-9dd3-890605cef49a'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:33.222345 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-82c65d73-fb60-4de8-9dd3-890605cef49a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6ddc880458ff4defa7e2ee9da419f258', 'request_id': 'req-82c65d73-fb60-4de8-9dd3-890605cef49a'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:33.229464 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-82c65d73-fb60-4de8-9dd3-890605cef49a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6ddc880458ff4defa7e2ee9da419f258', 'request_id': 'req-82c65d73-fb60-4de8-9dd3-890605cef49a'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:33.245670 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d649e27-ff91-4d0e-8b2a-6286248c3823 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:02:33.247980 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-82c65d73-fb60-4de8-9dd3-890605cef49a admin admin] The token cache is being invalidate because project 6ddc880458ff4defa7e2ee9da419f258 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80153) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 16:02:33.248509 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-82c65d73-fb60-4de8-9dd3-890605cef49a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:33.249607 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-82c65d73-fb60-4de8-9dd3-890605cef49a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:33.252124 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1067/2107] 10.222.0.46 () {58 vars in 1232 bytes} [Thu Jun 23 16:02:33 2022] DELETE /identity/v3/projects/6ddc880458ff4defa7e2ee9da419f258 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 16:02:33.256206 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe998e1-50e1-44e0-b627-cdeeb7903db4 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:33.256206 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe998e1-50e1-44e0-b627-cdeeb7903db4 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:33.256206 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe998e1-50e1-44e0-b627-cdeeb7903db4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:33.290314 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-6fe998e1-50e1-44e0-b627-cdeeb7903db4 None None] MFA Rules not processed for user `727f426107294de7a850170b93936e42`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:02:33.291563 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-6fe998e1-50e1-44e0-b627-cdeeb7903db4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:02:33.317963 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6ae05d-c916-4e4d-8269-9d7039533700 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:02:33.318904 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6ae05d-c916-4e4d-8269-9d7039533700 admin admin] REQUEST_METHOD: `PUT` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:33.319190 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6ae05d-c916-4e4d-8269-9d7039533700 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:33.319471 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6ae05d-c916-4e4d-8269-9d7039533700 admin admin] PATH_INFO: `/v3/projects/22de9db3dd79472a8451eb09127f67e9/users/a47620c49e10426d8c114a2f21671811/roles/e2a5bbf3200a4d4091d72bc7ff78c34d` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:33.333837 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6ae05d-c916-4e4d-8269-9d7039533700 admin admin] RBAC: Authorizing `identity:create_grant(project_id=22de9db3dd79472a8451eb09127f67e9, user_id=a47620c49e10426d8c114a2f21671811, role_id=e2a5bbf3200a4d4091d72bc7ff78c34d)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:02:33.337100 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6ae05d-c916-4e4d-8269-9d7039533700 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:02:33.356713 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1041/2108] 10.222.0.46 () {60 vars in 1482 bytes} [Thu Jun 23 16:02:33 2022] PUT /identity/v3/projects/22de9db3dd79472a8451eb09127f67e9/users/a47620c49e10426d8c114a2f21671811/roles/e2a5bbf3200a4d4091d72bc7ff78c34d => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 16:02:33.357119 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6ae05d-c916-4e4d-8269-9d7039533700 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:02:33.358796 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-9e6ae05d-c916-4e4d-8269-9d7039533700 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:02:33.374955 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1068/2109] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:02:33 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 16:02:33.376761 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8867d63d-9827-4845-abeb-cab9c8b89dcc None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:33.377030 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8867d63d-9827-4845-abeb-cab9c8b89dcc None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:33.377465 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8867d63d-9827-4845-abeb-cab9c8b89dcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:33.386635 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:02:33.416062 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-8867d63d-9827-4845-abeb-cab9c8b89dcc None None] MFA Rules not processed for user `a47620c49e10426d8c114a2f21671811`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:02:33.417233 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-8867d63d-9827-4845-abeb-cab9c8b89dcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:02:33.437733 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6ae05d-c916-4e4d-8269-9d7039533700 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:02:33.443193 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39db20bd-b3ce-44ba-afd9-834b75bae6f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:02:33.444091 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39db20bd-b3ce-44ba-afd9-834b75bae6f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:33.444384 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39db20bd-b3ce-44ba-afd9-834b75bae6f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:33.444687 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39db20bd-b3ce-44ba-afd9-834b75bae6f4 admin admin] PATH_INFO: `/v3/users/c8be0a3c70824636bf1235cc1db8b8c8` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:33.445950 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39db20bd-b3ce-44ba-afd9-834b75bae6f4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c8be0a3c70824636bf1235cc1db8b8c8)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:02:33.447057 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:02:33.447997 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39db20bd-b3ce-44ba-afd9-834b75bae6f4 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:02:33.496146 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1069/2110] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:02:33 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 16:02:33.496995 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8867d63d-9827-4845-abeb-cab9c8b89dcc None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:02:33.498466 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-8867d63d-9827-4845-abeb-cab9c8b89dcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:02:33.504398 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-39db20bd-b3ce-44ba-afd9-834b75bae6f4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c8be0a3c70824636bf1235cc1db8b8c8'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:33.505080 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-39db20bd-b3ce-44ba-afd9-834b75bae6f4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c8be0a3c70824636bf1235cc1db8b8c8'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:33.512857 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-39db20bd-b3ce-44ba-afd9-834b75bae6f4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c8be0a3c70824636bf1235cc1db8b8c8'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:33.513508 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8867d63d-9827-4845-abeb-cab9c8b89dcc None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:02:33.519984 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e23ec9-1648-47d2-9929-0d126d1b74c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:02:33.521024 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e23ec9-1648-47d2-9929-0d126d1b74c4 admin admin] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:33.521342 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e23ec9-1648-47d2-9929-0d126d1b74c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:33.521693 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e23ec9-1648-47d2-9929-0d126d1b74c4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:33.521959 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-39db20bd-b3ce-44ba-afd9-834b75bae6f4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c8be0a3c70824636bf1235cc1db8b8c8'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:33.522440 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e23ec9-1648-47d2-9929-0d126d1b74c4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:02:33.522601 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-39db20bd-b3ce-44ba-afd9-834b75bae6f4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c8be0a3c70824636bf1235cc1db8b8c8'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:33.524939 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e23ec9-1648-47d2-9929-0d126d1b74c4 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:02:33.542962 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1042/2111] 10.222.0.46 () {58 vars in 1223 bytes} [Thu Jun 23 16:02:33 2022] DELETE /identity/v3/users/c8be0a3c70824636bf1235cc1db8b8c8 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 16:02:33.543554 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39db20bd-b3ce-44ba-afd9-834b75bae6f4 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:02:33.545313 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-39db20bd-b3ce-44ba-afd9-834b75bae6f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:02:33.561343 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1070/2112] 10.222.0.46 () {60 vars in 1152 bytes} [Thu Jun 23 16:02:33 2022] POST /identity/v3/projects => generated 367 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 16:02:33.562885 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e23ec9-1648-47d2-9929-0d126d1b74c4 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:02:33.564664 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-e9e23ec9-1648-47d2-9929-0d126d1b74c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:02:33.640818 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e23ec9-1648-47d2-9929-0d126d1b74c4 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:02:33.642249 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39db20bd-b3ce-44ba-afd9-834b75bae6f4 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:02:33.647578 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f820be7-dd6f-417e-838b-4a5dd501ca50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:02:33.648384 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38698324-660f-4796-b9e3-88fd5c9190d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:02:33.648536 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f820be7-dd6f-417e-838b-4a5dd501ca50 admin admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:33.649614 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38698324-660f-4796-b9e3-88fd5c9190d8 admin admin] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:33.649845 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f820be7-dd6f-417e-838b-4a5dd501ca50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:33.649966 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38698324-660f-4796-b9e3-88fd5c9190d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:33.650445 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38698324-660f-4796-b9e3-88fd5c9190d8 admin admin] PATH_INFO: `/v3/users` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:33.650658 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f820be7-dd6f-417e-838b-4a5dd501ca50 admin admin] PATH_INFO: `/v3/projects` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:33.651176 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38698324-660f-4796-b9e3-88fd5c9190d8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:02:33.651634 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f820be7-dd6f-417e-838b-4a5dd501ca50 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:02:33.653351 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38698324-660f-4796-b9e3-88fd5c9190d8 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:02:33.653814 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f820be7-dd6f-417e-838b-4a5dd501ca50 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:02:33.680449 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1071/2113] 10.222.0.46 () {60 vars in 1143 bytes} [Thu Jun 23 16:02:33 2022] POST /identity/v3/users => generated 290 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 16:02:33.683222 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38698324-660f-4796-b9e3-88fd5c9190d8 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:02:33.691417 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1043/2114] 10.222.0.46 () {60 vars in 1152 bytes} [Thu Jun 23 16:02:33 2022] POST /identity/v3/projects => generated 399 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 16:02:33.694724 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38698324-660f-4796-b9e3-88fd5c9190d8 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:02:33.696265 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f820be7-dd6f-417e-838b-4a5dd501ca50 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:02:33.729199 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:02:33.771609 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e8f036-e10f-4d3c-aeb4-4772a11180f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:02:33.772633 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e8f036-e10f-4d3c-aeb4-4772a11180f2 admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:33.773066 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e8f036-e10f-4d3c-aeb4-4772a11180f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:33.773514 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e8f036-e10f-4d3c-aeb4-4772a11180f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:33.774905 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e8f036-e10f-4d3c-aeb4-4772a11180f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:02:33.776883 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e8f036-e10f-4d3c-aeb4-4772a11180f2 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:02:33.787589 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f820be7-dd6f-417e-838b-4a5dd501ca50 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:02:33.792707 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1072/2115] 10.222.0.46 () {58 vars in 1121 bytes} [Thu Jun 23 16:02:33 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 16:02:33.793625 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc592eb-ec0e-464a-b88f-0cfcb32e42e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:02:33.794146 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e8f036-e10f-4d3c-aeb4-4772a11180f2 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:02:33.794847 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc592eb-ec0e-464a-b88f-0cfcb32e42e9 admin admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:33.795145 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc592eb-ec0e-464a-b88f-0cfcb32e42e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:33.796055 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc592eb-ec0e-464a-b88f-0cfcb32e42e9 admin admin] PATH_INFO: `/v3/users` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:33.797243 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc592eb-ec0e-464a-b88f-0cfcb32e42e9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:02:33.799371 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc592eb-ec0e-464a-b88f-0cfcb32e42e9 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:02:33.805763 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e8f036-e10f-4d3c-aeb4-4772a11180f2 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:02:33.811948 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe6107f-bacb-4190-b98b-70e56b3883b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:02:33.812862 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe6107f-bacb-4190-b98b-70e56b3883b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:33.813146 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe6107f-bacb-4190-b98b-70e56b3883b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:33.813424 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe6107f-bacb-4190-b98b-70e56b3883b6 admin admin] PATH_INFO: `/v3/projects/b7a941ee69ef4c409a3f3f03aa6701cf/users/f419cb1b4fee458aa42c705ce752db2f/roles/812d3c567c52469cb2b35709fbdc857c` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:33.825868 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1044/2116] 10.222.0.46 () {60 vars in 1143 bytes} [Thu Jun 23 16:02:33 2022] POST /identity/v3/users => generated 306 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 16:02:33.830524 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc592eb-ec0e-464a-b88f-0cfcb32e42e9 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:02:33.834975 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe6107f-bacb-4190-b98b-70e56b3883b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b7a941ee69ef4c409a3f3f03aa6701cf, user_id=f419cb1b4fee458aa42c705ce752db2f, role_id=812d3c567c52469cb2b35709fbdc857c)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:02:33.838266 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe6107f-bacb-4190-b98b-70e56b3883b6 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:02:33.843009 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc592eb-ec0e-464a-b88f-0cfcb32e42e9 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:02:33.850495 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4639d5-0ecc-49ac-93d6-eb2a43d23cba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:02:33.851281 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4639d5-0ecc-49ac-93d6-eb2a43d23cba admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:33.852229 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4639d5-0ecc-49ac-93d6-eb2a43d23cba admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:33.852229 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4639d5-0ecc-49ac-93d6-eb2a43d23cba admin admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:33.852722 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4639d5-0ecc-49ac-93d6-eb2a43d23cba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:02:33.854580 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4639d5-0ecc-49ac-93d6-eb2a43d23cba admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:02:33.858995 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1073/2117] 10.222.0.46 () {60 vars in 1482 bytes} [Thu Jun 23 16:02:33 2022] PUT /identity/v3/projects/b7a941ee69ef4c409a3f3f03aa6701cf/users/f419cb1b4fee458aa42c705ce752db2f/roles/812d3c567c52469cb2b35709fbdc857c => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 16:02:33.863064 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe6107f-bacb-4190-b98b-70e56b3883b6 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:02:33.873418 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1045/2118] 10.222.0.46 () {58 vars in 1121 bytes} [Thu Jun 23 16:02:33 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 16:02:33.873623 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:02:33.875422 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4639d5-0ecc-49ac-93d6-eb2a43d23cba admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:02:33.890558 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4639d5-0ecc-49ac-93d6-eb2a43d23cba admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:02:33.957496 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe6107f-bacb-4190-b98b-70e56b3883b6 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:02:33.963410 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214845a9-a591-427d-bd50-62ef3a8fe309 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:02:33.964145 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214845a9-a591-427d-bd50-62ef3a8fe309 admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:33.964433 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214845a9-a591-427d-bd50-62ef3a8fe309 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:33.964718 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214845a9-a591-427d-bd50-62ef3a8fe309 admin admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:33.965466 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214845a9-a591-427d-bd50-62ef3a8fe309 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:02:33.967215 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214845a9-a591-427d-bd50-62ef3a8fe309 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:02:33.969109 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e955a3-3907-4b58-9636-f87addf44208 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:02:33.970023 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e955a3-3907-4b58-9636-f87addf44208 admin admin] REQUEST_METHOD: `PUT` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:33.970305 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e955a3-3907-4b58-9636-f87addf44208 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:33.970577 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e955a3-3907-4b58-9636-f87addf44208 admin admin] PATH_INFO: `/v3/projects/1eb87dd85b4e4c4da7cc69460f63fbd5/users/5926454f010345218b70031e3fe9e81f/roles/812d3c567c52469cb2b35709fbdc857c` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:33.978480 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1074/2119] 10.222.0.46 () {58 vars in 1121 bytes} [Thu Jun 23 16:02:33 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 16:02:33.979240 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214845a9-a591-427d-bd50-62ef3a8fe309 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:02:33.982943 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-214845a9-a591-427d-bd50-62ef3a8fe309 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:02:33.990778 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e955a3-3907-4b58-9636-f87addf44208 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1eb87dd85b4e4c4da7cc69460f63fbd5, user_id=5926454f010345218b70031e3fe9e81f, role_id=812d3c567c52469cb2b35709fbdc857c)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:02:33.994364 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e955a3-3907-4b58-9636-f87addf44208 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:02:33.996989 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214845a9-a591-427d-bd50-62ef3a8fe309 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:02:34.002176 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f22fd14-3acf-44e0-a1d5-8e72082e2b8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:02:34.003683 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f22fd14-3acf-44e0-a1d5-8e72082e2b8a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:34.003990 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f22fd14-3acf-44e0-a1d5-8e72082e2b8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:34.004363 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f22fd14-3acf-44e0-a1d5-8e72082e2b8a admin admin] PATH_INFO: `/v3/projects/d3f70497554e4dacb1f7d35cb19e409d` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:34.005762 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f22fd14-3acf-44e0-a1d5-8e72082e2b8a admin admin] RBAC: Authorizing `identity:delete_project(project_id=d3f70497554e4dacb1f7d35cb19e409d)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:02:34.007935 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f22fd14-3acf-44e0-a1d5-8e72082e2b8a admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:02:34.011353 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1046/2120] 10.222.0.46 () {60 vars in 1482 bytes} [Thu Jun 23 16:02:33 2022] PUT /identity/v3/projects/1eb87dd85b4e4c4da7cc69460f63fbd5/users/5926454f010345218b70031e3fe9e81f/roles/812d3c567c52469cb2b35709fbdc857c => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 16:02:34.012185 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e955a3-3907-4b58-9636-f87addf44208 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:02:34.073949 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-6f22fd14-3acf-44e0-a1d5-8e72082e2b8a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd3f70497554e4dacb1f7d35cb19e409d', 'request_id': 'req-6f22fd14-3acf-44e0-a1d5-8e72082e2b8a'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:34.080354 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-6f22fd14-3acf-44e0-a1d5-8e72082e2b8a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd3f70497554e4dacb1f7d35cb19e409d', 'request_id': 'req-6f22fd14-3acf-44e0-a1d5-8e72082e2b8a'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:34.088086 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-6f22fd14-3acf-44e0-a1d5-8e72082e2b8a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd3f70497554e4dacb1f7d35cb19e409d', 'request_id': 'req-6f22fd14-3acf-44e0-a1d5-8e72082e2b8a'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:34.110521 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e955a3-3907-4b58-9636-f87addf44208 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:02:34.114111 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-6f22fd14-3acf-44e0-a1d5-8e72082e2b8a admin admin] The token cache is being invalidate because project d3f70497554e4dacb1f7d35cb19e409d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80153) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 16:02:34.114796 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-6f22fd14-3acf-44e0-a1d5-8e72082e2b8a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:34.115519 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-6f22fd14-3acf-44e0-a1d5-8e72082e2b8a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:34.118170 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1075/2121] 10.222.0.46 () {58 vars in 1232 bytes} [Thu Jun 23 16:02:33 2022] DELETE /identity/v3/projects/d3f70497554e4dacb1f7d35cb19e409d => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 16:02:34.118827 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f22fd14-3acf-44e0-a1d5-8e72082e2b8a admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:02:34.120540 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-6f22fd14-3acf-44e0-a1d5-8e72082e2b8a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:02:34.153563 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:02:34.189712 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca49398-149a-4800-a690-18aeb3fa7a26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:02:34.190576 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca49398-149a-4800-a690-18aeb3fa7a26 admin admin] REQUEST_METHOD: `PUT` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:34.190896 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca49398-149a-4800-a690-18aeb3fa7a26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:34.191171 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca49398-149a-4800-a690-18aeb3fa7a26 admin admin] PATH_INFO: `/v3/projects/b7a941ee69ef4c409a3f3f03aa6701cf/users/f419cb1b4fee458aa42c705ce752db2f/roles/812d3c567c52469cb2b35709fbdc857c` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:34.193188 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca49398-149a-4800-a690-18aeb3fa7a26 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b7a941ee69ef4c409a3f3f03aa6701cf, user_id=f419cb1b4fee458aa42c705ce752db2f, role_id=812d3c567c52469cb2b35709fbdc857c)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:02:34.195961 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca49398-149a-4800-a690-18aeb3fa7a26 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:02:34.203761 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f22fd14-3acf-44e0-a1d5-8e72082e2b8a admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:02:34.207291 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1047/2122] 10.222.0.46 () {60 vars in 1482 bytes} [Thu Jun 23 16:02:34 2022] PUT /identity/v3/projects/b7a941ee69ef4c409a3f3f03aa6701cf/users/f419cb1b4fee458aa42c705ce752db2f/roles/812d3c567c52469cb2b35709fbdc857c => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 16:02:34.208315 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca49398-149a-4800-a690-18aeb3fa7a26 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:02:34.209982 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-aca49398-149a-4800-a690-18aeb3fa7a26 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:02:34.240249 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:02:34.272347 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f3880b-b3e7-4fea-804f-95ad7a79b06e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:02:34.273031 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f3880b-b3e7-4fea-804f-95ad7a79b06e admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:34.273319 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f3880b-b3e7-4fea-804f-95ad7a79b06e admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:34.274131 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f3880b-b3e7-4fea-804f-95ad7a79b06e admin admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:34.274875 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f3880b-b3e7-4fea-804f-95ad7a79b06e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:02:34.276754 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f3880b-b3e7-4fea-804f-95ad7a79b06e admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:02:34.285485 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca49398-149a-4800-a690-18aeb3fa7a26 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:02:34.291430 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1076/2123] 10.222.0.46 () {58 vars in 1121 bytes} [Thu Jun 23 16:02:34 2022] GET /identity/v3/roles => generated 1420 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 16:02:34.291430 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938e9ea2-7b7a-4241-9743-4fbb17940a2d None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:34.291430 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938e9ea2-7b7a-4241-9743-4fbb17940a2d None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:34.291430 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938e9ea2-7b7a-4241-9743-4fbb17940a2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:34.295623 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa627270-d37a-4bfb-9d61-70f483af7593 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:02:34.296682 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa627270-d37a-4bfb-9d61-70f483af7593 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:34.296973 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa627270-d37a-4bfb-9d61-70f483af7593 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:34.297289 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa627270-d37a-4bfb-9d61-70f483af7593 admin admin] PATH_INFO: `/v3/users/53351ae192c4491ab543810d4d85937a` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:34.298551 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa627270-d37a-4bfb-9d61-70f483af7593 admin admin] RBAC: Authorizing `identity:delete_user(user_id=53351ae192c4491ab543810d4d85937a)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:02:34.300642 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa627270-d37a-4bfb-9d61-70f483af7593 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:02:34.320064 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-938e9ea2-7b7a-4241-9743-4fbb17940a2d None None] MFA Rules not processed for user `f419cb1b4fee458aa42c705ce752db2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:02:34.321524 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-938e9ea2-7b7a-4241-9743-4fbb17940a2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:02:34.362428 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-fa627270-d37a-4bfb-9d61-70f483af7593 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '53351ae192c4491ab543810d4d85937a'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:34.363200 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-fa627270-d37a-4bfb-9d61-70f483af7593 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '53351ae192c4491ab543810d4d85937a'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:34.370974 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-fa627270-d37a-4bfb-9d61-70f483af7593 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '53351ae192c4491ab543810d4d85937a'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:34.381337 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-fa627270-d37a-4bfb-9d61-70f483af7593 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '53351ae192c4491ab543810d4d85937a'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:34.381934 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-fa627270-d37a-4bfb-9d61-70f483af7593 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '53351ae192c4491ab543810d4d85937a'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:34.403087 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1048/2124] 10.222.0.46 () {58 vars in 1223 bytes} [Thu Jun 23 16:02:34 2022] DELETE /identity/v3/users/53351ae192c4491ab543810d4d85937a => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 16:02:34.404779 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa627270-d37a-4bfb-9d61-70f483af7593 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:02:34.407079 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-fa627270-d37a-4bfb-9d61-70f483af7593 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:02:34.415600 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1077/2125] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:02:34 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 16:02:34.433185 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938e9ea2-7b7a-4241-9743-4fbb17940a2d None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:02:34.435125 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-938e9ea2-7b7a-4241-9743-4fbb17940a2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:02:34.514229 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:02:34.542231 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa627270-d37a-4bfb-9d61-70f483af7593 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:02:34.547709 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745bffad-24ad-4c05-89d8-0fc9758bdb2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:02:34.548637 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745bffad-24ad-4c05-89d8-0fc9758bdb2e admin admin] REQUEST_METHOD: `PUT` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:34.548931 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745bffad-24ad-4c05-89d8-0fc9758bdb2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:34.549217 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745bffad-24ad-4c05-89d8-0fc9758bdb2e admin admin] PATH_INFO: `/v3/projects/1eb87dd85b4e4c4da7cc69460f63fbd5/users/5926454f010345218b70031e3fe9e81f/roles/812d3c567c52469cb2b35709fbdc857c` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:34.551570 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745bffad-24ad-4c05-89d8-0fc9758bdb2e admin admin] RBAC: Authorizing `identity:create_grant(project_id=1eb87dd85b4e4c4da7cc69460f63fbd5, user_id=5926454f010345218b70031e3fe9e81f, role_id=812d3c567c52469cb2b35709fbdc857c)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:02:34.554584 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745bffad-24ad-4c05-89d8-0fc9758bdb2e admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:02:34.559533 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938e9ea2-7b7a-4241-9743-4fbb17940a2d None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:02:34.568095 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1049/2126] 10.222.0.46 () {60 vars in 1482 bytes} [Thu Jun 23 16:02:34 2022] PUT /identity/v3/projects/1eb87dd85b4e4c4da7cc69460f63fbd5/users/5926454f010345218b70031e3fe9e81f/roles/812d3c567c52469cb2b35709fbdc857c => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 16:02:34.580535 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e33708-8017-4fcc-89f2-1d563078619e None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:34.580970 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e33708-8017-4fcc-89f2-1d563078619e None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:34.581243 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e33708-8017-4fcc-89f2-1d563078619e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:34.635060 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-a6e33708-8017-4fcc-89f2-1d563078619e None None] MFA Rules not processed for user `5926454f010345218b70031e3fe9e81f`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:02:34.636653 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-a6e33708-8017-4fcc-89f2-1d563078619e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:02:34.663426 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65df5fe-8ad5-4331-a975-64f6ff0b829e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce18eaaba3a6477684fff1fd98ec8b51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:02:34.663999 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65df5fe-8ad5-4331-a975-64f6ff0b829e service cinder] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:34.664275 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65df5fe-8ad5-4331-a975-64f6ff0b829e service cinder] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:34.664554 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65df5fe-8ad5-4331-a975-64f6ff0b829e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:34.691731 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:02:34.749073 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1050/2127] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:02:34 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 16:02:34.757700 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e33708-8017-4fcc-89f2-1d563078619e None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:02:34.759839 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-a6e33708-8017-4fcc-89f2-1d563078619e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:02:34.762245 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65df5fe-8ad5-4331-a975-64f6ff0b829e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:02:34.764523 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65df5fe-8ad5-4331-a975-64f6ff0b829e service cinder] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:02:34.776255 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1078/2128] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jun 23 16:02:34 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 16:02:34.777116 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65df5fe-8ad5-4331-a975-64f6ff0b829e service cinder] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:02:34.778544 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-b65df5fe-8ad5-4331-a975-64f6ff0b829e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:02:34.792782 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65df5fe-8ad5-4331-a975-64f6ff0b829e service cinder] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:02:34.796701 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f1b662-314a-4091-86d5-92afd8b1325c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce18eaaba3a6477684fff1fd98ec8b51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:02:34.797514 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f1b662-314a-4091-86d5-92afd8b1325c service cinder] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:34.797800 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f1b662-314a-4091-86d5-92afd8b1325c service cinder] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:34.798069 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f1b662-314a-4091-86d5-92afd8b1325c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:34.808839 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f1b662-314a-4091-86d5-92afd8b1325c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:02:34.810290 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f1b662-314a-4091-86d5-92afd8b1325c service cinder] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:02:34.825514 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1079/2129] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jun 23 16:02:34 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 16:02:34.851219 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e33708-8017-4fcc-89f2-1d563078619e None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:02:34.856150 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f436c1-5910-4a46-8975-7eb30371fbad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:02:34.857079 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f436c1-5910-4a46-8975-7eb30371fbad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:34.857429 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f436c1-5910-4a46-8975-7eb30371fbad admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:34.857800 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f436c1-5910-4a46-8975-7eb30371fbad admin admin] PATH_INFO: `/v3/projects/eb6459aaf3794fe5b4bed757bbe3bfbf` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:34.863750 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f436c1-5910-4a46-8975-7eb30371fbad admin admin] RBAC: Authorizing `identity:delete_project(project_id=eb6459aaf3794fe5b4bed757bbe3bfbf)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:02:34.865928 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f436c1-5910-4a46-8975-7eb30371fbad admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:02:34.871362 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f1b662-314a-4091-86d5-92afd8b1325c service cinder] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:02:34.883867 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f1b662-314a-4091-86d5-92afd8b1325c service cinder] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:02:34.888661 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df97d7df-cf68-4e78-9d60-b55689e1a897 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce18eaaba3a6477684fff1fd98ec8b51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:02:34.889307 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df97d7df-cf68-4e78-9d60-b55689e1a897 service cinder] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:34.889766 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df97d7df-cf68-4e78-9d60-b55689e1a897 service cinder] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:34.890045 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df97d7df-cf68-4e78-9d60-b55689e1a897 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:34.892356 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-df97d7df-cf68-4e78-9d60-b55689e1a897 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:02:34.927604 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-b2f436c1-5910-4a46-8975-7eb30371fbad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eb6459aaf3794fe5b4bed757bbe3bfbf', 'request_id': 'req-b2f436c1-5910-4a46-8975-7eb30371fbad'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:34.934289 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-b2f436c1-5910-4a46-8975-7eb30371fbad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eb6459aaf3794fe5b4bed757bbe3bfbf', 'request_id': 'req-b2f436c1-5910-4a46-8975-7eb30371fbad'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:34.942432 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-b2f436c1-5910-4a46-8975-7eb30371fbad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eb6459aaf3794fe5b4bed757bbe3bfbf', 'request_id': 'req-b2f436c1-5910-4a46-8975-7eb30371fbad'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:34.943909 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:02:34.959180 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df97d7df-cf68-4e78-9d60-b55689e1a897 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:02:34.959373 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-b2f436c1-5910-4a46-8975-7eb30371fbad admin admin] The token cache is being invalidate because project eb6459aaf3794fe5b4bed757bbe3bfbf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80154) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 16:02:34.959501 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-b2f436c1-5910-4a46-8975-7eb30371fbad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:34.960164 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-b2f436c1-5910-4a46-8975-7eb30371fbad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:34.960624 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df97d7df-cf68-4e78-9d60-b55689e1a897 service cinder] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:02:34.961819 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1051/2130] 10.222.0.46 () {58 vars in 1232 bytes} [Thu Jun 23 16:02:34 2022] DELETE /identity/v3/projects/eb6459aaf3794fe5b4bed757bbe3bfbf => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 16:02:34.965593 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f436c1-5910-4a46-8975-7eb30371fbad admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:02:34.965593 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-b2f436c1-5910-4a46-8975-7eb30371fbad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:02:34.970374 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1080/2131] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jun 23 16:02:34 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 16:02:34.971745 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2350d1f-c29b-42cc-a98b-94fcaccd8f4f None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:34.972039 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2350d1f-c29b-42cc-a98b-94fcaccd8f4f None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:34.972316 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2350d1f-c29b-42cc-a98b-94fcaccd8f4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:34.990184 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-b2350d1f-c29b-42cc-a98b-94fcaccd8f4f None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:02:34.991497 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-b2350d1f-c29b-42cc-a98b-94fcaccd8f4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:02:35.036597 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f436c1-5910-4a46-8975-7eb30371fbad admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:02:35.042053 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6be3e46-b6e1-4ae7-92d0-4062660bd44b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce18eaaba3a6477684fff1fd98ec8b51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:02:35.042741 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6be3e46-b6e1-4ae7-92d0-4062660bd44b service cinder] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:35.043033 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6be3e46-b6e1-4ae7-92d0-4062660bd44b service cinder] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:35.043314 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6be3e46-b6e1-4ae7-92d0-4062660bd44b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:35.045917 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-a6be3e46-b6e1-4ae7-92d0-4062660bd44b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:02:35.078796 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1081/2132] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:02:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 16:02:35.087187 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9191cd8-301e-4b39-b632-699da0072308 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:35.088041 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9191cd8-301e-4b39-b632-699da0072308 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:35.088041 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9191cd8-301e-4b39-b632-699da0072308 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:35.110381 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-f9191cd8-301e-4b39-b632-699da0072308 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:02:35.111699 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-f9191cd8-301e-4b39-b632-699da0072308 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:02:35.122307 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1082/2133] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:02:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 16:02:35.128163 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9191cd8-301e-4b39-b632-699da0072308 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:02:35.132865 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6be3e46-b6e1-4ae7-92d0-4062660bd44b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:02:35.135318 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6be3e46-b6e1-4ae7-92d0-4062660bd44b service cinder] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:02:35.139028 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9191cd8-301e-4b39-b632-699da0072308 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:02:35.145851 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426fd266-e7c9-44d9-a7f8-5b9733c5dfd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:02:35.146534 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1052/2134] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jun 23 16:02:34 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 16:02:35.149007 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426fd266-e7c9-44d9-a7f8-5b9733c5dfd2 admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:35.149399 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426fd266-e7c9-44d9-a7f8-5b9733c5dfd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:35.149591 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426fd266-e7c9-44d9-a7f8-5b9733c5dfd2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:35.150783 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426fd266-e7c9-44d9-a7f8-5b9733c5dfd2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 16:02:35.151217 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426fd266-e7c9-44d9-a7f8-5b9733c5dfd2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:02:35.153577 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426fd266-e7c9-44d9-a7f8-5b9733c5dfd2 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:02:35.168763 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1083/2135] 10.222.0.46 () {58 vars in 1152 bytes} [Thu Jun 23 16:02:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 16:02:35.173487 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6be3e46-b6e1-4ae7-92d0-4062660bd44b service cinder] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:02:35.188312 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6be3e46-b6e1-4ae7-92d0-4062660bd44b service cinder] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:02:35.193494 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b385fb-da50-40cd-a723-177194aa3172 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:02:35.194147 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b385fb-da50-40cd-a723-177194aa3172 admin admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:35.194439 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b385fb-da50-40cd-a723-177194aa3172 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:35.194738 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b385fb-da50-40cd-a723-177194aa3172 admin admin] PATH_INFO: `/v3/projects` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:35.195521 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b385fb-da50-40cd-a723-177194aa3172 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:02:35.198180 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b385fb-da50-40cd-a723-177194aa3172 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:02:35.237036 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1053/2136] 10.222.0.46 () {60 vars in 1152 bytes} [Thu Jun 23 16:02:35 2022] POST /identity/v3/projects => generated 373 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 16:02:35.242323 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426fd266-e7c9-44d9-a7f8-5b9733c5dfd2 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:02:35.259233 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426fd266-e7c9-44d9-a7f8-5b9733c5dfd2 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:02:35.331924 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:02:35.399689 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080f6925-9151-4cac-ae6c-36338cd90730 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:02:35.400814 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080f6925-9151-4cac-ae6c-36338cd90730 admin admin] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:35.401896 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080f6925-9151-4cac-ae6c-36338cd90730 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:35.401896 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080f6925-9151-4cac-ae6c-36338cd90730 admin admin] PATH_INFO: `/v3/users` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:35.402474 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080f6925-9151-4cac-ae6c-36338cd90730 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:02:35.405553 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080f6925-9151-4cac-ae6c-36338cd90730 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:02:35.438634 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1084/2137] 10.222.0.46 () {60 vars in 1143 bytes} [Thu Jun 23 16:02:35 2022] POST /identity/v3/users => generated 287 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 16:02:35.442410 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b385fb-da50-40cd-a723-177194aa3172 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:02:35.455574 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b385fb-da50-40cd-a723-177194aa3172 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:02:35.461641 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640c5f5e-a751-463c-b1af-33ec2bf3eee6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:02:35.462318 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640c5f5e-a751-463c-b1af-33ec2bf3eee6 admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:35.462606 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640c5f5e-a751-463c-b1af-33ec2bf3eee6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:35.462917 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640c5f5e-a751-463c-b1af-33ec2bf3eee6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:35.463797 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640c5f5e-a751-463c-b1af-33ec2bf3eee6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:02:35.465804 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640c5f5e-a751-463c-b1af-33ec2bf3eee6 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:02:35.480294 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1054/2138] 10.222.0.46 () {58 vars in 1121 bytes} [Thu Jun 23 16:02:35 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 16:02:35.483437 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080f6925-9151-4cac-ae6c-36338cd90730 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:02:35.496688 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080f6925-9151-4cac-ae6c-36338cd90730 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:02:35.502352 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973daf5c-e9de-40d0-8a40-a64bf2cae975 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:02:35.503256 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973daf5c-e9de-40d0-8a40-a64bf2cae975 admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:35.503519 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973daf5c-e9de-40d0-8a40-a64bf2cae975 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:35.503983 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973daf5c-e9de-40d0-8a40-a64bf2cae975 admin admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:35.504791 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973daf5c-e9de-40d0-8a40-a64bf2cae975 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:02:35.506822 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973daf5c-e9de-40d0-8a40-a64bf2cae975 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:02:35.524310 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1085/2139] 10.222.0.46 () {58 vars in 1121 bytes} [Thu Jun 23 16:02:35 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 16:02:35.528959 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640c5f5e-a751-463c-b1af-33ec2bf3eee6 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:02:35.541877 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640c5f5e-a751-463c-b1af-33ec2bf3eee6 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:02:35.557162 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59e0503-c1dc-473d-836f-06631545b88c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:02:35.558922 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59e0503-c1dc-473d-836f-06631545b88c admin admin] REQUEST_METHOD: `PUT` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:35.560407 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59e0503-c1dc-473d-836f-06631545b88c admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:35.560707 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59e0503-c1dc-473d-836f-06631545b88c admin admin] PATH_INFO: `/v3/projects/ab9c00b7758f45daa7063ee2327c59d0/users/ad8dac501e254a8d9671083b7b7d413c/roles/e2a5bbf3200a4d4091d72bc7ff78c34d` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:35.577538 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59e0503-c1dc-473d-836f-06631545b88c admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab9c00b7758f45daa7063ee2327c59d0, user_id=ad8dac501e254a8d9671083b7b7d413c, role_id=e2a5bbf3200a4d4091d72bc7ff78c34d)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:02:35.580866 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59e0503-c1dc-473d-836f-06631545b88c admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:02:35.595024 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1055/2140] 10.222.0.46 () {60 vars in 1482 bytes} [Thu Jun 23 16:02:35 2022] PUT /identity/v3/projects/ab9c00b7758f45daa7063ee2327c59d0/users/ad8dac501e254a8d9671083b7b7d413c/roles/e2a5bbf3200a4d4091d72bc7ff78c34d => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 16:02:35.600426 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31f0e02-d343-49cd-aebc-59fa709f92b4 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:35.600924 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31f0e02-d343-49cd-aebc-59fa709f92b4 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:35.601037 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31f0e02-d343-49cd-aebc-59fa709f92b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:35.619676 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-e31f0e02-d343-49cd-aebc-59fa709f92b4 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:02:35.620876 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-e31f0e02-d343-49cd-aebc-59fa709f92b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:02:35.680657 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1086/2141] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:02:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 16:02:35.690942 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59e0503-c1dc-473d-836f-06631545b88c admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:02:35.692664 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-c59e0503-c1dc-473d-836f-06631545b88c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:02:35.760792 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59e0503-c1dc-473d-836f-06631545b88c admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:02:35.765006 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20448099-f2f6-41e1-9a12-1c109e4aefeb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51784064951a4619ae09b9167fb0fd0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:02:35.765683 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20448099-f2f6-41e1-9a12-1c109e4aefeb service neutron] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:35.765970 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20448099-f2f6-41e1-9a12-1c109e4aefeb service neutron] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:35.766250 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20448099-f2f6-41e1-9a12-1c109e4aefeb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:35.772701 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:02:35.779201 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20448099-f2f6-41e1-9a12-1c109e4aefeb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:02:35.782089 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20448099-f2f6-41e1-9a12-1c109e4aefeb service neutron] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:02:35.801142 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1056/2142] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jun 23 16:02:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 16:02:36.409774 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54ac079-74d8-4f2e-a961-1030bed4f8ea None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:36.409868 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54ac079-74d8-4f2e-a961-1030bed4f8ea None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:36.410607 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54ac079-74d8-4f2e-a961-1030bed4f8ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:36.446137 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-d54ac079-74d8-4f2e-a961-1030bed4f8ea None None] MFA Rules not processed for user `ad8dac501e254a8d9671083b7b7d413c`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:02:36.446137 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-d54ac079-74d8-4f2e-a961-1030bed4f8ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:02:36.535685 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1087/2143] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:02:36 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 16:02:36.547930 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20448099-f2f6-41e1-9a12-1c109e4aefeb service neutron] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:02:36.549361 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-20448099-f2f6-41e1-9a12-1c109e4aefeb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:02:36.668828 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20448099-f2f6-41e1-9a12-1c109e4aefeb service neutron] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:02:36.673942 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1839fd3-94db-4762-b2e1-edc5bb8f7a5e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce18eaaba3a6477684fff1fd98ec8b51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:02:36.674740 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1839fd3-94db-4762-b2e1-edc5bb8f7a5e service cinder] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:36.675036 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1839fd3-94db-4762-b2e1-edc5bb8f7a5e service cinder] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:36.675310 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1839fd3-94db-4762-b2e1-edc5bb8f7a5e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:36.686093 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1839fd3-94db-4762-b2e1-edc5bb8f7a5e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:02:36.687592 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1839fd3-94db-4762-b2e1-edc5bb8f7a5e service cinder] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:02:36.698342 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1057/2144] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jun 23 16:02:36 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 16:02:41.254872 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67ae08d-0ee2-4680-91d4-e0d2cdb76a9a None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:41.255342 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67ae08d-0ee2-4680-91d4-e0d2cdb76a9a None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:41.255445 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67ae08d-0ee2-4680-91d4-e0d2cdb76a9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:41.276280 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-f67ae08d-0ee2-4680-91d4-e0d2cdb76a9a None None] MFA Rules not processed for user `ad8dac501e254a8d9671083b7b7d413c`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:02:41.289116 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-f67ae08d-0ee2-4680-91d4-e0d2cdb76a9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:02:41.303072 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1088/2145] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:02:41 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 16:02:42.813288 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1839fd3-94db-4762-b2e1-edc5bb8f7a5e service cinder] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:02:42.815064 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-f1839fd3-94db-4762-b2e1-edc5bb8f7a5e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:02:42.829514 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1839fd3-94db-4762-b2e1-edc5bb8f7a5e service cinder] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:02:42.834410 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7591051d-38e6-4aa6-9651-0407c0472760 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:02:42.835349 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7591051d-38e6-4aa6-9651-0407c0472760 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:42.835632 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7591051d-38e6-4aa6-9651-0407c0472760 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:42.835914 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7591051d-38e6-4aa6-9651-0407c0472760 admin admin] PATH_INFO: `/v3/users/a47620c49e10426d8c114a2f21671811` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:42.837244 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7591051d-38e6-4aa6-9651-0407c0472760 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a47620c49e10426d8c114a2f21671811)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:02:42.839242 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7591051d-38e6-4aa6-9651-0407c0472760 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:02:42.893735 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-7591051d-38e6-4aa6-9651-0407c0472760 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a47620c49e10426d8c114a2f21671811'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:42.894576 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-7591051d-38e6-4aa6-9651-0407c0472760 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a47620c49e10426d8c114a2f21671811'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:42.903901 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-7591051d-38e6-4aa6-9651-0407c0472760 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a47620c49e10426d8c114a2f21671811'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:42.912695 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-7591051d-38e6-4aa6-9651-0407c0472760 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a47620c49e10426d8c114a2f21671811'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:42.913338 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-7591051d-38e6-4aa6-9651-0407c0472760 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a47620c49e10426d8c114a2f21671811'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:42.932659 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1058/2146] 10.222.0.46 () {58 vars in 1223 bytes} [Thu Jun 23 16:02:42 2022] DELETE /identity/v3/users/a47620c49e10426d8c114a2f21671811 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 16:02:42.937224 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03b1d9f-6add-4620-b9e4-36a9086ca42f None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:42.937739 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03b1d9f-6add-4620-b9e4-36a9086ca42f None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:42.937852 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03b1d9f-6add-4620-b9e4-36a9086ca42f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:42.956273 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-e03b1d9f-6add-4620-b9e4-36a9086ca42f None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:02:42.957533 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-e03b1d9f-6add-4620-b9e4-36a9086ca42f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:02:42.981534 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:02:43.029753 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1089/2147] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:02:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 16:02:43.040385 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7591051d-38e6-4aa6-9651-0407c0472760 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:02:43.041808 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-7591051d-38e6-4aa6-9651-0407c0472760 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:02:43.104789 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7591051d-38e6-4aa6-9651-0407c0472760 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:02:43.108689 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884dfb20-0b31-478d-b8ca-9910a2f90edc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51784064951a4619ae09b9167fb0fd0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:02:43.109354 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884dfb20-0b31-478d-b8ca-9910a2f90edc service neutron] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:43.109645 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884dfb20-0b31-478d-b8ca-9910a2f90edc service neutron] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:43.109926 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884dfb20-0b31-478d-b8ca-9910a2f90edc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:43.119742 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884dfb20-0b31-478d-b8ca-9910a2f90edc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:02:43.121197 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884dfb20-0b31-478d-b8ca-9910a2f90edc service neutron] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:02:43.131405 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1059/2148] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jun 23 16:02:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 16:02:43.432733 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03b1d9f-6add-4620-b9e4-36a9086ca42f None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:02:43.434292 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-e03b1d9f-6add-4620-b9e4-36a9086ca42f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:02:43.447939 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03b1d9f-6add-4620-b9e4-36a9086ca42f None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:02:43.452725 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519059c3-36f9-4465-a582-3fdfac1e2379 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:02:43.453610 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519059c3-36f9-4465-a582-3fdfac1e2379 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:43.453893 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519059c3-36f9-4465-a582-3fdfac1e2379 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:43.454176 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519059c3-36f9-4465-a582-3fdfac1e2379 admin admin] PATH_INFO: `/v3/projects/22de9db3dd79472a8451eb09127f67e9` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:43.455444 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519059c3-36f9-4465-a582-3fdfac1e2379 admin admin] RBAC: Authorizing `identity:delete_project(project_id=22de9db3dd79472a8451eb09127f67e9)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:02:43.457409 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519059c3-36f9-4465-a582-3fdfac1e2379 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:02:43.507074 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-519059c3-36f9-4465-a582-3fdfac1e2379 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '22de9db3dd79472a8451eb09127f67e9', 'request_id': 'req-519059c3-36f9-4465-a582-3fdfac1e2379'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:43.513166 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-519059c3-36f9-4465-a582-3fdfac1e2379 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '22de9db3dd79472a8451eb09127f67e9', 'request_id': 'req-519059c3-36f9-4465-a582-3fdfac1e2379'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:43.519803 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-519059c3-36f9-4465-a582-3fdfac1e2379 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '22de9db3dd79472a8451eb09127f67e9', 'request_id': 'req-519059c3-36f9-4465-a582-3fdfac1e2379'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:43.534220 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-519059c3-36f9-4465-a582-3fdfac1e2379 admin admin] The token cache is being invalidate because project 22de9db3dd79472a8451eb09127f67e9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80153) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 16:02:43.534518 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-519059c3-36f9-4465-a582-3fdfac1e2379 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:43.535219 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-519059c3-36f9-4465-a582-3fdfac1e2379 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:43.537747 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1090/2149] 10.222.0.46 () {58 vars in 1232 bytes} [Thu Jun 23 16:02:43 2022] DELETE /identity/v3/projects/22de9db3dd79472a8451eb09127f67e9 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 16:02:43.541886 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884dfb20-0b31-478d-b8ca-9910a2f90edc service neutron] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:02:43.552248 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884dfb20-0b31-478d-b8ca-9910a2f90edc service neutron] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:02:43.561520 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:02:43.622868 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eafa276-eeaa-4b8b-975e-ba269f47210b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:02:43.623778 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eafa276-eeaa-4b8b-975e-ba269f47210b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:43.624069 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eafa276-eeaa-4b8b-975e-ba269f47210b admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:43.624353 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eafa276-eeaa-4b8b-975e-ba269f47210b admin admin] PATH_INFO: `/v3/users/5926454f010345218b70031e3fe9e81f` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:43.625648 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eafa276-eeaa-4b8b-975e-ba269f47210b admin admin] RBAC: Authorizing `identity:delete_user(user_id=5926454f010345218b70031e3fe9e81f)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:02:43.627866 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eafa276-eeaa-4b8b-975e-ba269f47210b admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:02:43.693863 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-4eafa276-eeaa-4b8b-975e-ba269f47210b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5926454f010345218b70031e3fe9e81f'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:43.694599 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-4eafa276-eeaa-4b8b-975e-ba269f47210b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5926454f010345218b70031e3fe9e81f'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:43.703161 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-4eafa276-eeaa-4b8b-975e-ba269f47210b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5926454f010345218b70031e3fe9e81f'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:43.713071 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-4eafa276-eeaa-4b8b-975e-ba269f47210b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5926454f010345218b70031e3fe9e81f'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:43.713701 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-4eafa276-eeaa-4b8b-975e-ba269f47210b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5926454f010345218b70031e3fe9e81f'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:43.734900 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1060/2150] 10.222.0.46 () {58 vars in 1223 bytes} [Thu Jun 23 16:02:43 2022] DELETE /identity/v3/users/5926454f010345218b70031e3fe9e81f => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 16:02:44.119072 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519059c3-36f9-4465-a582-3fdfac1e2379 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:02:44.120653 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-519059c3-36f9-4465-a582-3fdfac1e2379 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:02:44.137150 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519059c3-36f9-4465-a582-3fdfac1e2379 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:02:44.199546 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8525370-1a34-4212-9fa4-d8d5f657ce98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:02:44.200448 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8525370-1a34-4212-9fa4-d8d5f657ce98 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:44.200755 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8525370-1a34-4212-9fa4-d8d5f657ce98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:44.201007 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8525370-1a34-4212-9fa4-d8d5f657ce98 admin admin] PATH_INFO: `/v3/projects/1eb87dd85b4e4c4da7cc69460f63fbd5` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:44.202212 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8525370-1a34-4212-9fa4-d8d5f657ce98 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1eb87dd85b4e4c4da7cc69460f63fbd5)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:02:44.204181 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8525370-1a34-4212-9fa4-d8d5f657ce98 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:02:44.260158 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-e8525370-1a34-4212-9fa4-d8d5f657ce98 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1eb87dd85b4e4c4da7cc69460f63fbd5', 'request_id': 'req-e8525370-1a34-4212-9fa4-d8d5f657ce98'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:44.267134 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:02:44.272270 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-e8525370-1a34-4212-9fa4-d8d5f657ce98 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1eb87dd85b4e4c4da7cc69460f63fbd5', 'request_id': 'req-e8525370-1a34-4212-9fa4-d8d5f657ce98'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:44.279670 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-e8525370-1a34-4212-9fa4-d8d5f657ce98 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1eb87dd85b4e4c4da7cc69460f63fbd5', 'request_id': 'req-e8525370-1a34-4212-9fa4-d8d5f657ce98'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:44.295123 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-e8525370-1a34-4212-9fa4-d8d5f657ce98 admin admin] The token cache is being invalidate because project 1eb87dd85b4e4c4da7cc69460f63fbd5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80153) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 16:02:44.295450 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-e8525370-1a34-4212-9fa4-d8d5f657ce98 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:44.296214 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-e8525370-1a34-4212-9fa4-d8d5f657ce98 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:44.298010 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1091/2151] 10.222.0.46 () {58 vars in 1232 bytes} [Thu Jun 23 16:02:44 2022] DELETE /identity/v3/projects/1eb87dd85b4e4c4da7cc69460f63fbd5 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 16:02:44.308253 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72157d1d-8baf-4d12-80f6-fb3b6fe2df75 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:44.308942 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72157d1d-8baf-4d12-80f6-fb3b6fe2df75 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:44.308942 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72157d1d-8baf-4d12-80f6-fb3b6fe2df75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:44.336068 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-72157d1d-8baf-4d12-80f6-fb3b6fe2df75 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:02:44.338315 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-72157d1d-8baf-4d12-80f6-fb3b6fe2df75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:02:44.434464 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1061/2152] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:02:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 16:02:44.443056 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e757dff9-823a-4514-97d3-7960e3c95d6c None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:44.443355 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e757dff9-823a-4514-97d3-7960e3c95d6c None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:44.443691 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e757dff9-823a-4514-97d3-7960e3c95d6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:44.464380 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-e757dff9-823a-4514-97d3-7960e3c95d6c None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:02:44.465607 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-e757dff9-823a-4514-97d3-7960e3c95d6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:02:44.473753 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1092/2153] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:02:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 16:02:44.478218 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72157d1d-8baf-4d12-80f6-fb3b6fe2df75 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:02:44.490902 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72157d1d-8baf-4d12-80f6-fb3b6fe2df75 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:02:44.497937 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044627b9-9b28-40e8-ac10-17d7660029f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:02:44.499014 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044627b9-9b28-40e8-ac10-17d7660029f9 admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:44.499452 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044627b9-9b28-40e8-ac10-17d7660029f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:44.499893 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044627b9-9b28-40e8-ac10-17d7660029f9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:44.501067 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044627b9-9b28-40e8-ac10-17d7660029f9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 16:02:44.501553 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044627b9-9b28-40e8-ac10-17d7660029f9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:02:44.504261 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044627b9-9b28-40e8-ac10-17d7660029f9 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:02:44.523162 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1062/2154] 10.222.0.46 () {58 vars in 1152 bytes} [Thu Jun 23 16:02:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 16:02:44.528388 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e757dff9-823a-4514-97d3-7960e3c95d6c None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:02:44.538483 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e757dff9-823a-4514-97d3-7960e3c95d6c None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:02:44.543513 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cae3b5e-f7e5-4967-afb0-54f212c67720 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:02:44.544189 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cae3b5e-f7e5-4967-afb0-54f212c67720 admin admin] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:44.544473 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cae3b5e-f7e5-4967-afb0-54f212c67720 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:44.544751 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cae3b5e-f7e5-4967-afb0-54f212c67720 admin admin] PATH_INFO: `/v3/projects` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:44.545473 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cae3b5e-f7e5-4967-afb0-54f212c67720 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:02:44.547488 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cae3b5e-f7e5-4967-afb0-54f212c67720 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:02:44.587676 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1093/2155] 10.222.0.46 () {60 vars in 1152 bytes} [Thu Jun 23 16:02:44 2022] POST /identity/v3/projects => generated 377 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 16:02:44.592149 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044627b9-9b28-40e8-ac10-17d7660029f9 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:02:44.603617 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044627b9-9b28-40e8-ac10-17d7660029f9 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:02:44.616747 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:02:44.673484 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e3a42b-4052-4fcd-81e7-16b3aeb086f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:02:44.674186 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e3a42b-4052-4fcd-81e7-16b3aeb086f1 admin admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:44.674471 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e3a42b-4052-4fcd-81e7-16b3aeb086f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:44.674800 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e3a42b-4052-4fcd-81e7-16b3aeb086f1 admin admin] PATH_INFO: `/v3/users` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:44.675520 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e3a42b-4052-4fcd-81e7-16b3aeb086f1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:02:44.677507 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e3a42b-4052-4fcd-81e7-16b3aeb086f1 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:02:44.699654 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1063/2156] 10.222.0.46 () {60 vars in 1143 bytes} [Thu Jun 23 16:02:44 2022] POST /identity/v3/users => generated 289 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 16:02:44.704567 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cae3b5e-f7e5-4967-afb0-54f212c67720 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:02:44.717332 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cae3b5e-f7e5-4967-afb0-54f212c67720 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:02:44.722916 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c4ce78-fe3f-4aee-9019-1c0495aadf5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:02:44.723269 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c4ce78-fe3f-4aee-9019-1c0495aadf5c admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:44.723635 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c4ce78-fe3f-4aee-9019-1c0495aadf5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:44.723973 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c4ce78-fe3f-4aee-9019-1c0495aadf5c admin admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:44.724909 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c4ce78-fe3f-4aee-9019-1c0495aadf5c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:02:44.726528 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c4ce78-fe3f-4aee-9019-1c0495aadf5c admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:02:44.739527 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1094/2157] 10.222.0.46 () {58 vars in 1121 bytes} [Thu Jun 23 16:02:44 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 16:02:44.744057 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e3a42b-4052-4fcd-81e7-16b3aeb086f1 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:02:44.755162 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e3a42b-4052-4fcd-81e7-16b3aeb086f1 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:02:44.760685 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2ec47e-1346-4b14-b70a-0d6f59c770ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:02:44.761222 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2ec47e-1346-4b14-b70a-0d6f59c770ca admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:44.761507 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2ec47e-1346-4b14-b70a-0d6f59c770ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:44.761784 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2ec47e-1346-4b14-b70a-0d6f59c770ca admin admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:44.762525 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2ec47e-1346-4b14-b70a-0d6f59c770ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:02:44.764192 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2ec47e-1346-4b14-b70a-0d6f59c770ca admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:02:44.775729 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1064/2158] 10.222.0.46 () {58 vars in 1121 bytes} [Thu Jun 23 16:02:44 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 16:02:44.780306 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c4ce78-fe3f-4aee-9019-1c0495aadf5c admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:02:44.791658 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c4ce78-fe3f-4aee-9019-1c0495aadf5c admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:02:44.796711 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9c6c8c-8913-4f21-b9f6-ac5dfdafd7bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:02:44.797623 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9c6c8c-8913-4f21-b9f6-ac5dfdafd7bc admin admin] REQUEST_METHOD: `PUT` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:44.797906 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9c6c8c-8913-4f21-b9f6-ac5dfdafd7bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:44.798185 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9c6c8c-8913-4f21-b9f6-ac5dfdafd7bc admin admin] PATH_INFO: `/v3/projects/e3adde0ad7a842819a990180a0cbfd52/users/2c5f445dfdf3452ab1570c9678de1432/roles/e2a5bbf3200a4d4091d72bc7ff78c34d` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:44.814072 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9c6c8c-8913-4f21-b9f6-ac5dfdafd7bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=e3adde0ad7a842819a990180a0cbfd52, user_id=2c5f445dfdf3452ab1570c9678de1432, role_id=e2a5bbf3200a4d4091d72bc7ff78c34d)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:02:44.816792 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9c6c8c-8913-4f21-b9f6-ac5dfdafd7bc admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:02:44.832550 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1095/2159] 10.222.0.46 () {60 vars in 1482 bytes} [Thu Jun 23 16:02:44 2022] PUT /identity/v3/projects/e3adde0ad7a842819a990180a0cbfd52/users/2c5f445dfdf3452ab1570c9678de1432/roles/e2a5bbf3200a4d4091d72bc7ff78c34d => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 16:02:44.844867 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90261d4b-6ce7-4355-aab9-9239fe9773db None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:44.845266 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90261d4b-6ce7-4355-aab9-9239fe9773db None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:44.845561 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90261d4b-6ce7-4355-aab9-9239fe9773db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:44.881834 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-90261d4b-6ce7-4355-aab9-9239fe9773db None None] MFA Rules not processed for user `2c5f445dfdf3452ab1570c9678de1432`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:02:44.883134 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-90261d4b-6ce7-4355-aab9-9239fe9773db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:02:44.976597 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1065/2160] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:02:44 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 16:02:44.981155 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9c6c8c-8913-4f21-b9f6-ac5dfdafd7bc admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:02:45.065234 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9c6c8c-8913-4f21-b9f6-ac5dfdafd7bc admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:02:45.070413 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946c8c31-a8c1-4960-89d5-ef6791674c07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:02:45.071657 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946c8c31-a8c1-4960-89d5-ef6791674c07 admin admin] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:45.071963 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946c8c31-a8c1-4960-89d5-ef6791674c07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:45.072239 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946c8c31-a8c1-4960-89d5-ef6791674c07 admin admin] PATH_INFO: `/v3/projects` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:45.072955 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946c8c31-a8c1-4960-89d5-ef6791674c07 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:02:45.075033 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946c8c31-a8c1-4960-89d5-ef6791674c07 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:02:45.114973 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1096/2161] 10.222.0.46 () {60 vars in 1152 bytes} [Thu Jun 23 16:02:44 2022] POST /identity/v3/projects => generated 377 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 16:02:45.118860 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90261d4b-6ce7-4355-aab9-9239fe9773db None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:02:45.131556 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90261d4b-6ce7-4355-aab9-9239fe9773db None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:02:45.197934 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b8b4bd-b197-4ae8-8b80-d9ad5dfe0af3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:02:45.198734 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b8b4bd-b197-4ae8-8b80-d9ad5dfe0af3 admin admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:45.199035 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b8b4bd-b197-4ae8-8b80-d9ad5dfe0af3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:45.199313 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b8b4bd-b197-4ae8-8b80-d9ad5dfe0af3 admin admin] PATH_INFO: `/v3/users` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:45.200072 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b8b4bd-b197-4ae8-8b80-d9ad5dfe0af3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:02:45.202243 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b8b4bd-b197-4ae8-8b80-d9ad5dfe0af3 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:02:45.232290 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1066/2162] 10.222.0.46 () {60 vars in 1143 bytes} [Thu Jun 23 16:02:45 2022] POST /identity/v3/users => generated 289 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 16:02:45.237266 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946c8c31-a8c1-4960-89d5-ef6791674c07 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:02:45.256560 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946c8c31-a8c1-4960-89d5-ef6791674c07 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:02:45.265661 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f873b5-db1c-4503-8cbf-9731bb63cabb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:02:45.266880 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f873b5-db1c-4503-8cbf-9731bb63cabb admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:45.267393 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f873b5-db1c-4503-8cbf-9731bb63cabb admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:45.267873 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f873b5-db1c-4503-8cbf-9731bb63cabb admin admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:45.269284 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f873b5-db1c-4503-8cbf-9731bb63cabb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:02:45.272437 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f873b5-db1c-4503-8cbf-9731bb63cabb admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:02:45.291947 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1097/2163] 10.222.0.46 () {58 vars in 1121 bytes} [Thu Jun 23 16:02:45 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 16:02:45.293279 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b8b4bd-b197-4ae8-8b80-d9ad5dfe0af3 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:02:45.306880 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b8b4bd-b197-4ae8-8b80-d9ad5dfe0af3 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:02:45.314362 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54239bb-685e-4bd8-8a85-c88fa987cfe5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:02:45.315517 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54239bb-685e-4bd8-8a85-c88fa987cfe5 admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:45.316053 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54239bb-685e-4bd8-8a85-c88fa987cfe5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:45.316669 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54239bb-685e-4bd8-8a85-c88fa987cfe5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:45.318419 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54239bb-685e-4bd8-8a85-c88fa987cfe5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:02:45.321623 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54239bb-685e-4bd8-8a85-c88fa987cfe5 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:02:45.341883 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1067/2164] 10.222.0.46 () {58 vars in 1121 bytes} [Thu Jun 23 16:02:45 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 16:02:45.346449 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f873b5-db1c-4503-8cbf-9731bb63cabb admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:02:45.357494 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f873b5-db1c-4503-8cbf-9731bb63cabb admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:02:45.362464 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd20685-7ef5-4616-be48-62b96a71eec0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:02:45.363427 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd20685-7ef5-4616-be48-62b96a71eec0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:45.363714 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd20685-7ef5-4616-be48-62b96a71eec0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:45.363996 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd20685-7ef5-4616-be48-62b96a71eec0 admin admin] PATH_INFO: `/v3/projects/ba9e454609a340169ec224346424651f/users/a069f4599ea24d49b681e3222d45e09f/roles/e2a5bbf3200a4d4091d72bc7ff78c34d` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:45.384456 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd20685-7ef5-4616-be48-62b96a71eec0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba9e454609a340169ec224346424651f, user_id=a069f4599ea24d49b681e3222d45e09f, role_id=e2a5bbf3200a4d4091d72bc7ff78c34d)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:02:45.387295 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd20685-7ef5-4616-be48-62b96a71eec0 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:02:45.402320 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1098/2165] 10.222.0.46 () {60 vars in 1482 bytes} [Thu Jun 23 16:02:45 2022] PUT /identity/v3/projects/ba9e454609a340169ec224346424651f/users/a069f4599ea24d49b681e3222d45e09f/roles/e2a5bbf3200a4d4091d72bc7ff78c34d => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 16:02:45.414099 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4b363f-6571-4163-aae6-bcbc6a6c915e None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:45.414450 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4b363f-6571-4163-aae6-bcbc6a6c915e None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:45.414836 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4b363f-6571-4163-aae6-bcbc6a6c915e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:45.421753 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:02:45.456060 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-1f4b363f-6571-4163-aae6-bcbc6a6c915e None None] MFA Rules not processed for user `a069f4599ea24d49b681e3222d45e09f`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:02:45.457646 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-1f4b363f-6571-4163-aae6-bcbc6a6c915e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:02:45.536996 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1068/2166] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:02:45 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 16:02:45.542320 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd20685-7ef5-4616-be48-62b96a71eec0 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:02:45.590155 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=93,keystone:DELETE=2,keystone:INSERT=5 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:02:45.631488 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd20685-7ef5-4616-be48-62b96a71eec0 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:02:45.637485 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac4bc73-603d-46b2-91b9-85914aedaf71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:02:45.638418 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac4bc73-603d-46b2-91b9-85914aedaf71 admin admin] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:45.638538 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac4bc73-603d-46b2-91b9-85914aedaf71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:45.638857 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac4bc73-603d-46b2-91b9-85914aedaf71 admin admin] PATH_INFO: `/v3/projects` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:45.639635 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac4bc73-603d-46b2-91b9-85914aedaf71 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:02:45.641758 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac4bc73-603d-46b2-91b9-85914aedaf71 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:02:45.672425 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1099/2167] 10.222.0.46 () {60 vars in 1152 bytes} [Thu Jun 23 16:02:45 2022] POST /identity/v3/projects => generated 379 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 16:02:45.677560 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4b363f-6571-4163-aae6-bcbc6a6c915e None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:02:45.691314 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4b363f-6571-4163-aae6-bcbc6a6c915e None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:02:45.766522 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48dc2eb2-95db-4b18-90eb-7c1060fede9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:02:45.767391 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dc2eb2-95db-4b18-90eb-7c1060fede9b admin admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:45.767683 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dc2eb2-95db-4b18-90eb-7c1060fede9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:45.767963 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dc2eb2-95db-4b18-90eb-7c1060fede9b admin admin] PATH_INFO: `/v3/users` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:45.768685 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48dc2eb2-95db-4b18-90eb-7c1060fede9b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:02:45.770683 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48dc2eb2-95db-4b18-90eb-7c1060fede9b admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:02:45.794041 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1069/2168] 10.222.0.46 () {60 vars in 1143 bytes} [Thu Jun 23 16:02:45 2022] POST /identity/v3/users => generated 296 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 16:02:45.798937 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac4bc73-603d-46b2-91b9-85914aedaf71 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:02:45.811260 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac4bc73-603d-46b2-91b9-85914aedaf71 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:02:45.816475 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ebfa88-1d55-41c1-98e1-2f36eee4b425 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:02:45.817264 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ebfa88-1d55-41c1-98e1-2f36eee4b425 admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:45.817547 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ebfa88-1d55-41c1-98e1-2f36eee4b425 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:45.817874 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ebfa88-1d55-41c1-98e1-2f36eee4b425 admin admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:45.818644 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ebfa88-1d55-41c1-98e1-2f36eee4b425 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:02:45.820338 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ebfa88-1d55-41c1-98e1-2f36eee4b425 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:02:45.833184 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1100/2169] 10.222.0.46 () {58 vars in 1121 bytes} [Thu Jun 23 16:02:45 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 16:02:45.837706 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48dc2eb2-95db-4b18-90eb-7c1060fede9b admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:02:45.848761 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48dc2eb2-95db-4b18-90eb-7c1060fede9b admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:02:45.854370 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8dbcf2b-a509-4043-862f-8abbe52a431d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:02:45.855322 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dbcf2b-a509-4043-862f-8abbe52a431d admin admin] REQUEST_METHOD: `PUT` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:45.855613 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dbcf2b-a509-4043-862f-8abbe52a431d admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:45.855895 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dbcf2b-a509-4043-862f-8abbe52a431d admin admin] PATH_INFO: `/v3/projects/59bbb0eed10847dda49906f85353f668/users/c6642b7bdc5e4804aa0923605872079c/roles/812d3c567c52469cb2b35709fbdc857c` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:45.873438 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8dbcf2b-a509-4043-862f-8abbe52a431d admin admin] RBAC: Authorizing `identity:create_grant(project_id=59bbb0eed10847dda49906f85353f668, user_id=c6642b7bdc5e4804aa0923605872079c, role_id=812d3c567c52469cb2b35709fbdc857c)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:02:45.876199 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8dbcf2b-a509-4043-862f-8abbe52a431d admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:02:45.892058 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1070/2170] 10.222.0.46 () {60 vars in 1482 bytes} [Thu Jun 23 16:02:45 2022] PUT /identity/v3/projects/59bbb0eed10847dda49906f85353f668/users/c6642b7bdc5e4804aa0923605872079c/roles/812d3c567c52469cb2b35709fbdc857c => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 16:02:45.897888 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ebfa88-1d55-41c1-98e1-2f36eee4b425 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:02:45.909920 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ebfa88-1d55-41c1-98e1-2f36eee4b425 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:02:45.975783 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fb0283-805d-4c23-afe5-5be0d9fc7cb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:02:45.976668 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fb0283-805d-4c23-afe5-5be0d9fc7cb0 admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:45.977485 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fb0283-805d-4c23-afe5-5be0d9fc7cb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:45.977744 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fb0283-805d-4c23-afe5-5be0d9fc7cb0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:45.978513 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fb0283-805d-4c23-afe5-5be0d9fc7cb0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:02:45.980176 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fb0283-805d-4c23-afe5-5be0d9fc7cb0 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:02:45.993905 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1101/2171] 10.222.0.46 () {58 vars in 1121 bytes} [Thu Jun 23 16:02:45 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 16:02:45.998246 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8dbcf2b-a509-4043-862f-8abbe52a431d admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:02:46.009081 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8dbcf2b-a509-4043-862f-8abbe52a431d admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:02:46.013987 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c18debc-0ebf-4fd2-ba99-e7f7494b4038 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:02:46.014944 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c18debc-0ebf-4fd2-ba99-e7f7494b4038 admin admin] REQUEST_METHOD: `PUT` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:46.015252 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c18debc-0ebf-4fd2-ba99-e7f7494b4038 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:46.015533 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c18debc-0ebf-4fd2-ba99-e7f7494b4038 admin admin] PATH_INFO: `/v3/projects/59bbb0eed10847dda49906f85353f668/users/c6642b7bdc5e4804aa0923605872079c/roles/812d3c567c52469cb2b35709fbdc857c` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:46.017645 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c18debc-0ebf-4fd2-ba99-e7f7494b4038 admin admin] RBAC: Authorizing `identity:create_grant(project_id=59bbb0eed10847dda49906f85353f668, user_id=c6642b7bdc5e4804aa0923605872079c, role_id=812d3c567c52469cb2b35709fbdc857c)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:02:46.020372 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c18debc-0ebf-4fd2-ba99-e7f7494b4038 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:02:46.032169 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1071/2172] 10.222.0.46 () {60 vars in 1482 bytes} [Thu Jun 23 16:02:45 2022] PUT /identity/v3/projects/59bbb0eed10847dda49906f85353f668/users/c6642b7bdc5e4804aa0923605872079c/roles/812d3c567c52469cb2b35709fbdc857c => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 16:02:46.043649 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f49dd4f-4ad8-4d57-a553-03c7a9815d8b None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:46.043932 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f49dd4f-4ad8-4d57-a553-03c7a9815d8b None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:46.044210 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f49dd4f-4ad8-4d57-a553-03c7a9815d8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:46.081743 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-8f49dd4f-4ad8-4d57-a553-03c7a9815d8b None None] MFA Rules not processed for user `c6642b7bdc5e4804aa0923605872079c`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:02:46.083733 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-8f49dd4f-4ad8-4d57-a553-03c7a9815d8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:02:46.186674 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1102/2173] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:02:46 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 16:02:46.199935 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c18debc-0ebf-4fd2-ba99-e7f7494b4038 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:02:46.218209 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-0c18debc-0ebf-4fd2-ba99-e7f7494b4038 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:02:46.284579 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:02:46.291229 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c18debc-0ebf-4fd2-ba99-e7f7494b4038 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:02:46.296172 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3044b766-4154-4e2d-bbef-3c45a26549d5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce18eaaba3a6477684fff1fd98ec8b51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:02:46.296839 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3044b766-4154-4e2d-bbef-3c45a26549d5 service cinder] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:46.297132 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3044b766-4154-4e2d-bbef-3c45a26549d5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:46.297408 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3044b766-4154-4e2d-bbef-3c45a26549d5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:46.308366 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3044b766-4154-4e2d-bbef-3c45a26549d5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:02:46.309692 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3044b766-4154-4e2d-bbef-3c45a26549d5 service cinder] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:02:46.320147 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1072/2174] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jun 23 16:02:46 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 16:02:46.380364 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f49dd4f-4ad8-4d57-a553-03c7a9815d8b None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:02:46.400000 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f49dd4f-4ad8-4d57-a553-03c7a9815d8b None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:02:46.403915 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c357140-5c27-4635-9ba5-85b90fae38c7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce18eaaba3a6477684fff1fd98ec8b51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:02:46.404578 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c357140-5c27-4635-9ba5-85b90fae38c7 service cinder] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:46.404856 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c357140-5c27-4635-9ba5-85b90fae38c7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:46.405171 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c357140-5c27-4635-9ba5-85b90fae38c7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:46.427341 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:02:46.474493 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c357140-5c27-4635-9ba5-85b90fae38c7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:02:46.476000 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c357140-5c27-4635-9ba5-85b90fae38c7 service cinder] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:02:46.486082 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1103/2175] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jun 23 16:02:46 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 16:02:50.819386 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3044b766-4154-4e2d-bbef-3c45a26549d5 service cinder] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:02:50.822036 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-3044b766-4154-4e2d-bbef-3c45a26549d5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:02:50.895253 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3044b766-4154-4e2d-bbef-3c45a26549d5 service cinder] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:02:50.901238 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3af7b9-a303-49f8-a07d-b4d4a86bc063 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:02:50.902233 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3af7b9-a303-49f8-a07d-b4d4a86bc063 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:50.902512 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3af7b9-a303-49f8-a07d-b4d4a86bc063 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:50.902848 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3af7b9-a303-49f8-a07d-b4d4a86bc063 admin admin] PATH_INFO: `/v3/users/727f426107294de7a850170b93936e42` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:50.904405 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3af7b9-a303-49f8-a07d-b4d4a86bc063 admin admin] RBAC: Authorizing `identity:delete_user(user_id=727f426107294de7a850170b93936e42)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:02:50.906733 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3af7b9-a303-49f8-a07d-b4d4a86bc063 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:02:50.985562 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-cd3af7b9-a303-49f8-a07d-b4d4a86bc063 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '727f426107294de7a850170b93936e42'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:50.986925 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-cd3af7b9-a303-49f8-a07d-b4d4a86bc063 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '727f426107294de7a850170b93936e42'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:51.010492 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-cd3af7b9-a303-49f8-a07d-b4d4a86bc063 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '727f426107294de7a850170b93936e42'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:51.026001 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-cd3af7b9-a303-49f8-a07d-b4d4a86bc063 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '727f426107294de7a850170b93936e42'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:51.027047 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-cd3af7b9-a303-49f8-a07d-b4d4a86bc063 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '727f426107294de7a850170b93936e42'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:51.048696 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1073/2176] 10.222.0.46 () {58 vars in 1223 bytes} [Thu Jun 23 16:02:50 2022] DELETE /identity/v3/users/727f426107294de7a850170b93936e42 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 16:02:51.055716 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144acda9-c68a-4d96-aa5b-5735666ae1b0 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:51.056229 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144acda9-c68a-4d96-aa5b-5735666ae1b0 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:51.056450 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144acda9-c68a-4d96-aa5b-5735666ae1b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:51.083942 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-144acda9-c68a-4d96-aa5b-5735666ae1b0 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:02:51.085493 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-144acda9-c68a-4d96-aa5b-5735666ae1b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:02:51.179249 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1104/2177] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:02:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 16:02:51.189691 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3af7b9-a303-49f8-a07d-b4d4a86bc063 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:02:51.191237 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-cd3af7b9-a303-49f8-a07d-b4d4a86bc063 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:02:51.270316 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3af7b9-a303-49f8-a07d-b4d4a86bc063 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:02:51.274060 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b9aa66-d787-485f-8f25-d87f65858507 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51784064951a4619ae09b9167fb0fd0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:02:51.274805 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b9aa66-d787-485f-8f25-d87f65858507 service neutron] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:51.275361 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b9aa66-d787-485f-8f25-d87f65858507 service neutron] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:51.275640 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b9aa66-d787-485f-8f25-d87f65858507 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:51.285335 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b9aa66-d787-485f-8f25-d87f65858507 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:02:51.286736 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b9aa66-d787-485f-8f25-d87f65858507 service neutron] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:02:51.297263 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1074/2178] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jun 23 16:02:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 16:02:51.631035 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144acda9-c68a-4d96-aa5b-5735666ae1b0 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:02:51.632692 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-144acda9-c68a-4d96-aa5b-5735666ae1b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:02:51.645877 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144acda9-c68a-4d96-aa5b-5735666ae1b0 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:02:51.653673 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179ffb9e-5962-4f61-af1f-3587ea909223 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:02:51.655562 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179ffb9e-5962-4f61-af1f-3587ea909223 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:51.655854 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179ffb9e-5962-4f61-af1f-3587ea909223 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:51.656154 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179ffb9e-5962-4f61-af1f-3587ea909223 admin admin] PATH_INFO: `/v3/users/2c5f445dfdf3452ab1570c9678de1432` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:51.657440 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179ffb9e-5962-4f61-af1f-3587ea909223 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2c5f445dfdf3452ab1570c9678de1432)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:02:51.659544 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179ffb9e-5962-4f61-af1f-3587ea909223 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:02:51.693013 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b9aa66-d787-485f-8f25-d87f65858507 service neutron] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:02:51.694660 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-c6b9aa66-d787-485f-8f25-d87f65858507 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:02:51.709449 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b9aa66-d787-485f-8f25-d87f65858507 service neutron] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:02:51.715245 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f95bae-0172-430b-96db-cea711ef7108 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:02:51.716203 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f95bae-0172-430b-96db-cea711ef7108 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:51.716714 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f95bae-0172-430b-96db-cea711ef7108 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:51.717375 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f95bae-0172-430b-96db-cea711ef7108 admin admin] PATH_INFO: `/v3/projects/4333d946851b4293a126a25934d6c355` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:51.718526 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f95bae-0172-430b-96db-cea711ef7108 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4333d946851b4293a126a25934d6c355)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:02:51.724299 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f95bae-0172-430b-96db-cea711ef7108 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:02:51.733559 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:02:51.744440 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-179ffb9e-5962-4f61-af1f-3587ea909223 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2c5f445dfdf3452ab1570c9678de1432'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:51.754079 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-179ffb9e-5962-4f61-af1f-3587ea909223 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2c5f445dfdf3452ab1570c9678de1432'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:51.755280 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-179ffb9e-5962-4f61-af1f-3587ea909223 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2c5f445dfdf3452ab1570c9678de1432'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:51.755973 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-179ffb9e-5962-4f61-af1f-3587ea909223 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2c5f445dfdf3452ab1570c9678de1432'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:51.773263 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-179ffb9e-5962-4f61-af1f-3587ea909223 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2c5f445dfdf3452ab1570c9678de1432'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:51.785950 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-16f95bae-0172-430b-96db-cea711ef7108 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4333d946851b4293a126a25934d6c355', 'request_id': 'req-16f95bae-0172-430b-96db-cea711ef7108'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:51.792751 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1105/2179] 10.222.0.46 () {58 vars in 1223 bytes} [Thu Jun 23 16:02:51 2022] DELETE /identity/v3/users/2c5f445dfdf3452ab1570c9678de1432 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 16:02:51.795365 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-16f95bae-0172-430b-96db-cea711ef7108 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4333d946851b4293a126a25934d6c355', 'request_id': 'req-16f95bae-0172-430b-96db-cea711ef7108'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:51.797143 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f128e00-a28e-4982-995d-299cdd6d4749 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:51.797442 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f128e00-a28e-4982-995d-299cdd6d4749 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:51.797715 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f128e00-a28e-4982-995d-299cdd6d4749 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:51.806853 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-16f95bae-0172-430b-96db-cea711ef7108 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4333d946851b4293a126a25934d6c355', 'request_id': 'req-16f95bae-0172-430b-96db-cea711ef7108'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:51.815663 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-3f128e00-a28e-4982-995d-299cdd6d4749 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:02:51.816976 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-3f128e00-a28e-4982-995d-299cdd6d4749 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:02:51.827534 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-16f95bae-0172-430b-96db-cea711ef7108 admin admin] The token cache is being invalidate because project 4333d946851b4293a126a25934d6c355 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80154) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 16:02:51.828091 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-16f95bae-0172-430b-96db-cea711ef7108 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:51.828644 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-16f95bae-0172-430b-96db-cea711ef7108 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:51.830225 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1075/2180] 10.222.0.46 () {58 vars in 1232 bytes} [Thu Jun 23 16:02:51 2022] DELETE /identity/v3/projects/4333d946851b4293a126a25934d6c355 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 16:02:51.834156 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f95bae-0172-430b-96db-cea711ef7108 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:02:51.835612 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-16f95bae-0172-430b-96db-cea711ef7108 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:02:51.848987 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:02:51.881641 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1106/2181] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:02:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 16:02:51.891897 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f128e00-a28e-4982-995d-299cdd6d4749 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:02:51.893365 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-3f128e00-a28e-4982-995d-299cdd6d4749 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:02:51.903172 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f95bae-0172-430b-96db-cea711ef7108 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:02:51.907862 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d645d4de-08a1-48a7-a01c-4e23d86bef49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:02:51.908748 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d645d4de-08a1-48a7-a01c-4e23d86bef49 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:51.909040 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d645d4de-08a1-48a7-a01c-4e23d86bef49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:51.909317 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d645d4de-08a1-48a7-a01c-4e23d86bef49 admin admin] PATH_INFO: `/v3/users/f419cb1b4fee458aa42c705ce752db2f` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:51.910540 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d645d4de-08a1-48a7-a01c-4e23d86bef49 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f419cb1b4fee458aa42c705ce752db2f)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:02:51.912524 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d645d4de-08a1-48a7-a01c-4e23d86bef49 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:02:51.953664 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f128e00-a28e-4982-995d-299cdd6d4749 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:02:51.957325 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577fee3e-116c-45ad-a760-f4158c055204 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51784064951a4619ae09b9167fb0fd0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:02:51.957980 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577fee3e-116c-45ad-a760-f4158c055204 service neutron] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:51.958266 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577fee3e-116c-45ad-a760-f4158c055204 service neutron] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:51.958543 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577fee3e-116c-45ad-a760-f4158c055204 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:51.961085 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-d645d4de-08a1-48a7-a01c-4e23d86bef49 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f419cb1b4fee458aa42c705ce752db2f'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:51.962348 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-d645d4de-08a1-48a7-a01c-4e23d86bef49 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f419cb1b4fee458aa42c705ce752db2f'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:51.968609 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577fee3e-116c-45ad-a760-f4158c055204 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:02:51.969772 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-d645d4de-08a1-48a7-a01c-4e23d86bef49 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f419cb1b4fee458aa42c705ce752db2f'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:51.970280 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577fee3e-116c-45ad-a760-f4158c055204 service neutron] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:02:51.977567 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-d645d4de-08a1-48a7-a01c-4e23d86bef49 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f419cb1b4fee458aa42c705ce752db2f'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:51.978339 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-d645d4de-08a1-48a7-a01c-4e23d86bef49 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f419cb1b4fee458aa42c705ce752db2f'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:51.982835 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1107/2182] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jun 23 16:02:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 16:02:51.997473 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1076/2183] 10.222.0.46 () {58 vars in 1223 bytes} [Thu Jun 23 16:02:51 2022] DELETE /identity/v3/users/f419cb1b4fee458aa42c705ce752db2f => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 16:02:52.368125 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577fee3e-116c-45ad-a760-f4158c055204 service neutron] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:02:52.369666 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-577fee3e-116c-45ad-a760-f4158c055204 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:02:52.393628 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577fee3e-116c-45ad-a760-f4158c055204 service neutron] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:02:52.462500 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d645d4de-08a1-48a7-a01c-4e23d86bef49 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:02:52.474958 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-d645d4de-08a1-48a7-a01c-4e23d86bef49 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:02:52.493085 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:02:52.501174 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc885cd-21a4-44f4-8ab0-74cd8ab363f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:02:52.505046 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc885cd-21a4-44f4-8ab0-74cd8ab363f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:52.505415 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc885cd-21a4-44f4-8ab0-74cd8ab363f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:52.505723 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc885cd-21a4-44f4-8ab0-74cd8ab363f0 admin admin] PATH_INFO: `/v3/projects/e3adde0ad7a842819a990180a0cbfd52` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:52.511011 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc885cd-21a4-44f4-8ab0-74cd8ab363f0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e3adde0ad7a842819a990180a0cbfd52)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:02:52.516070 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc885cd-21a4-44f4-8ab0-74cd8ab363f0 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:02:52.609191 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d645d4de-08a1-48a7-a01c-4e23d86bef49 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:02:52.618950 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-5bc885cd-21a4-44f4-8ab0-74cd8ab363f0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e3adde0ad7a842819a990180a0cbfd52', 'request_id': 'req-5bc885cd-21a4-44f4-8ab0-74cd8ab363f0'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:52.625421 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-5bc885cd-21a4-44f4-8ab0-74cd8ab363f0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e3adde0ad7a842819a990180a0cbfd52', 'request_id': 'req-5bc885cd-21a4-44f4-8ab0-74cd8ab363f0'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:52.633564 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-5bc885cd-21a4-44f4-8ab0-74cd8ab363f0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e3adde0ad7a842819a990180a0cbfd52', 'request_id': 'req-5bc885cd-21a4-44f4-8ab0-74cd8ab363f0'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:52.647534 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-5bc885cd-21a4-44f4-8ab0-74cd8ab363f0 admin admin] The token cache is being invalidate because project e3adde0ad7a842819a990180a0cbfd52 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80153) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 16:02:52.647767 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-5bc885cd-21a4-44f4-8ab0-74cd8ab363f0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:52.648301 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-5bc885cd-21a4-44f4-8ab0-74cd8ab363f0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:52.650623 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1108/2184] 10.222.0.46 () {58 vars in 1232 bytes} [Thu Jun 23 16:02:52 2022] DELETE /identity/v3/projects/e3adde0ad7a842819a990180a0cbfd52 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 16:02:52.653932 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc885cd-21a4-44f4-8ab0-74cd8ab363f0 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:02:52.655913 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-5bc885cd-21a4-44f4-8ab0-74cd8ab363f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:02:52.669553 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:02:52.699916 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb82755-0e6e-4463-a7f2-f6e2840c9c7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:02:52.700945 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb82755-0e6e-4463-a7f2-f6e2840c9c7d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:52.701136 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb82755-0e6e-4463-a7f2-f6e2840c9c7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:52.701419 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb82755-0e6e-4463-a7f2-f6e2840c9c7d admin admin] PATH_INFO: `/v3/projects/b7a941ee69ef4c409a3f3f03aa6701cf` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:52.702901 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb82755-0e6e-4463-a7f2-f6e2840c9c7d admin admin] RBAC: Authorizing `identity:delete_project(project_id=b7a941ee69ef4c409a3f3f03aa6701cf)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:02:52.704825 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb82755-0e6e-4463-a7f2-f6e2840c9c7d admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:02:52.734506 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc885cd-21a4-44f4-8ab0-74cd8ab363f0 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:02:52.739700 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad4e3d2-83d6-4410-96f7-b4c6d86991a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:02:52.740620 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad4e3d2-83d6-4410-96f7-b4c6d86991a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:52.740906 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad4e3d2-83d6-4410-96f7-b4c6d86991a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:52.741179 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad4e3d2-83d6-4410-96f7-b4c6d86991a5 admin admin] PATH_INFO: `/v3/users/a069f4599ea24d49b681e3222d45e09f` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:52.742447 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad4e3d2-83d6-4410-96f7-b4c6d86991a5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a069f4599ea24d49b681e3222d45e09f)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:02:52.745028 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad4e3d2-83d6-4410-96f7-b4c6d86991a5 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:02:52.755907 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-bfb82755-0e6e-4463-a7f2-f6e2840c9c7d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b7a941ee69ef4c409a3f3f03aa6701cf', 'request_id': 'req-bfb82755-0e6e-4463-a7f2-f6e2840c9c7d'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:52.761531 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-bfb82755-0e6e-4463-a7f2-f6e2840c9c7d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b7a941ee69ef4c409a3f3f03aa6701cf', 'request_id': 'req-bfb82755-0e6e-4463-a7f2-f6e2840c9c7d'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:52.768435 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-bfb82755-0e6e-4463-a7f2-f6e2840c9c7d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b7a941ee69ef4c409a3f3f03aa6701cf', 'request_id': 'req-bfb82755-0e6e-4463-a7f2-f6e2840c9c7d'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:52.781593 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-bfb82755-0e6e-4463-a7f2-f6e2840c9c7d admin admin] The token cache is being invalidate because project b7a941ee69ef4c409a3f3f03aa6701cf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80154) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 16:02:52.782008 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-bfb82755-0e6e-4463-a7f2-f6e2840c9c7d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:52.782745 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-bfb82755-0e6e-4463-a7f2-f6e2840c9c7d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:52.784614 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1077/2185] 10.222.0.46 () {58 vars in 1232 bytes} [Thu Jun 23 16:02:52 2022] DELETE /identity/v3/projects/b7a941ee69ef4c409a3f3f03aa6701cf => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 16:02:52.790329 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cff3d1-19ca-488a-89ef-de53d72e30c3 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:52.790932 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cff3d1-19ca-488a-89ef-de53d72e30c3 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:52.791230 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cff3d1-19ca-488a-89ef-de53d72e30c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:52.808792 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-1ad4e3d2-83d6-4410-96f7-b4c6d86991a5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a069f4599ea24d49b681e3222d45e09f'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:52.810438 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-a8cff3d1-19ca-488a-89ef-de53d72e30c3 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:02:52.811690 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-a8cff3d1-19ca-488a-89ef-de53d72e30c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:02:52.816125 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-1ad4e3d2-83d6-4410-96f7-b4c6d86991a5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a069f4599ea24d49b681e3222d45e09f'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:52.816742 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-1ad4e3d2-83d6-4410-96f7-b4c6d86991a5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a069f4599ea24d49b681e3222d45e09f'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:52.817438 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-1ad4e3d2-83d6-4410-96f7-b4c6d86991a5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a069f4599ea24d49b681e3222d45e09f'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:52.830825 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-1ad4e3d2-83d6-4410-96f7-b4c6d86991a5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a069f4599ea24d49b681e3222d45e09f'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:52.846104 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1109/2186] 10.222.0.46 () {58 vars in 1223 bytes} [Thu Jun 23 16:02:52 2022] DELETE /identity/v3/users/a069f4599ea24d49b681e3222d45e09f => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 16:02:52.863305 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:02:52.881645 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1078/2187] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:02:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 16:02:52.890574 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b19bc2-863d-4c1b-95b6-72d606ebf9b4 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:52.891019 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b19bc2-863d-4c1b-95b6-72d606ebf9b4 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:52.891298 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b19bc2-863d-4c1b-95b6-72d606ebf9b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:52.912525 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-33b19bc2-863d-4c1b-95b6-72d606ebf9b4 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:02:52.913805 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-33b19bc2-863d-4c1b-95b6-72d606ebf9b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:02:52.985944 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1110/2188] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:02:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 16:02:52.990587 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cff3d1-19ca-488a-89ef-de53d72e30c3 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:02:53.003216 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cff3d1-19ca-488a-89ef-de53d72e30c3 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:02:53.009497 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f0410a-c692-4992-a01c-f234bbe5756e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:02:53.010316 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f0410a-c692-4992-a01c-f234bbe5756e admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:53.010519 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f0410a-c692-4992-a01c-f234bbe5756e admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:53.010988 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f0410a-c692-4992-a01c-f234bbe5756e admin admin] PATH_INFO: `/v3/domains` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:53.011771 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f0410a-c692-4992-a01c-f234bbe5756e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 16:02:53.012058 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f0410a-c692-4992-a01c-f234bbe5756e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:02:53.014413 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f0410a-c692-4992-a01c-f234bbe5756e admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:02:53.027716 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1079/2189] 10.222.0.46 () {58 vars in 1152 bytes} [Thu Jun 23 16:02:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 16:02:53.033454 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b19bc2-863d-4c1b-95b6-72d606ebf9b4 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:02:53.046062 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b19bc2-863d-4c1b-95b6-72d606ebf9b4 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:02:53.051359 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752144bb-e2f0-4e27-b017-862c28bcf6c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:02:53.052039 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752144bb-e2f0-4e27-b017-862c28bcf6c1 admin admin] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:53.052327 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752144bb-e2f0-4e27-b017-862c28bcf6c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:53.052609 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752144bb-e2f0-4e27-b017-862c28bcf6c1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:53.053315 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752144bb-e2f0-4e27-b017-862c28bcf6c1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:02:53.055344 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752144bb-e2f0-4e27-b017-862c28bcf6c1 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:02:53.085715 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1111/2190] 10.222.0.46 () {60 vars in 1152 bytes} [Thu Jun 23 16:02:53 2022] POST /identity/v3/projects => generated 361 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 16:02:53.087065 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f0410a-c692-4992-a01c-f234bbe5756e admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:02:53.097828 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f0410a-c692-4992-a01c-f234bbe5756e admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:02:53.141452 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752144bb-e2f0-4e27-b017-862c28bcf6c1 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:02:53.143244 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-752144bb-e2f0-4e27-b017-862c28bcf6c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:02:53.152345 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:02:53.171238 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89b6f96-df15-483e-ba33-0d2ceb03eaf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:02:53.171803 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89b6f96-df15-483e-ba33-0d2ceb03eaf1 admin admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:53.172140 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89b6f96-df15-483e-ba33-0d2ceb03eaf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:53.172355 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89b6f96-df15-483e-ba33-0d2ceb03eaf1 admin admin] PATH_INFO: `/v3/users` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:53.173056 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89b6f96-df15-483e-ba33-0d2ceb03eaf1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:02:53.175031 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89b6f96-df15-483e-ba33-0d2ceb03eaf1 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:02:53.197366 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1080/2191] 10.222.0.46 () {60 vars in 1143 bytes} [Thu Jun 23 16:02:53 2022] POST /identity/v3/users => generated 281 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 16:02:53.201257 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89b6f96-df15-483e-ba33-0d2ceb03eaf1 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:02:53.211767 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89b6f96-df15-483e-ba33-0d2ceb03eaf1 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:02:53.216472 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7fbb94-acea-4eb0-bd6a-ed669aab9304 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:02:53.217141 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7fbb94-acea-4eb0-bd6a-ed669aab9304 admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:53.217434 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7fbb94-acea-4eb0-bd6a-ed669aab9304 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:53.217710 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7fbb94-acea-4eb0-bd6a-ed669aab9304 admin admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:53.218561 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7fbb94-acea-4eb0-bd6a-ed669aab9304 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:02:53.220201 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7fbb94-acea-4eb0-bd6a-ed669aab9304 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:02:53.227525 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752144bb-e2f0-4e27-b017-862c28bcf6c1 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:02:53.230599 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1081/2192] 10.222.0.46 () {58 vars in 1121 bytes} [Thu Jun 23 16:02:53 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 16:02:53.234978 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7fbb94-acea-4eb0-bd6a-ed669aab9304 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:02:53.235505 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8756e497-2d41-4359-a8da-2e668d86348a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:02:53.236548 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8756e497-2d41-4359-a8da-2e668d86348a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:53.236837 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8756e497-2d41-4359-a8da-2e668d86348a admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:53.237252 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8756e497-2d41-4359-a8da-2e668d86348a admin admin] PATH_INFO: `/v3/projects/ba9e454609a340169ec224346424651f` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:53.238478 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8756e497-2d41-4359-a8da-2e668d86348a admin admin] RBAC: Authorizing `identity:delete_project(project_id=ba9e454609a340169ec224346424651f)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:02:53.240445 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8756e497-2d41-4359-a8da-2e668d86348a admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:02:53.246229 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7fbb94-acea-4eb0-bd6a-ed669aab9304 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:02:53.254207 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6168738-04f5-4623-975f-7761a7ea1f73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:02:53.254912 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6168738-04f5-4623-975f-7761a7ea1f73 admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:53.255203 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6168738-04f5-4623-975f-7761a7ea1f73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:53.255484 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6168738-04f5-4623-975f-7761a7ea1f73 admin admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:53.256221 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6168738-04f5-4623-975f-7761a7ea1f73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:02:53.257836 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6168738-04f5-4623-975f-7761a7ea1f73 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:02:53.272751 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1082/2193] 10.222.0.46 () {58 vars in 1121 bytes} [Thu Jun 23 16:02:53 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 16:02:53.272751 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6168738-04f5-4623-975f-7761a7ea1f73 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:02:53.274598 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-a6168738-04f5-4623-975f-7761a7ea1f73 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:02:53.287367 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6168738-04f5-4623-975f-7761a7ea1f73 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:02:53.292882 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-8756e497-2d41-4359-a8da-2e668d86348a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ba9e454609a340169ec224346424651f', 'request_id': 'req-8756e497-2d41-4359-a8da-2e668d86348a'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:53.298780 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-8756e497-2d41-4359-a8da-2e668d86348a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ba9e454609a340169ec224346424651f', 'request_id': 'req-8756e497-2d41-4359-a8da-2e668d86348a'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:53.305863 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-8756e497-2d41-4359-a8da-2e668d86348a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ba9e454609a340169ec224346424651f', 'request_id': 'req-8756e497-2d41-4359-a8da-2e668d86348a'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:53.319280 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-8756e497-2d41-4359-a8da-2e668d86348a admin admin] The token cache is being invalidate because project ba9e454609a340169ec224346424651f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80153) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 16:02:53.319570 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-8756e497-2d41-4359-a8da-2e668d86348a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:53.320249 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-8756e497-2d41-4359-a8da-2e668d86348a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:53.322406 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1112/2194] 10.222.0.46 () {58 vars in 1232 bytes} [Thu Jun 23 16:02:53 2022] DELETE /identity/v3/projects/ba9e454609a340169ec224346424651f => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 16:02:53.325416 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8756e497-2d41-4359-a8da-2e668d86348a admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:02:53.325416 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-8756e497-2d41-4359-a8da-2e668d86348a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:02:53.349448 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41773549-c7e2-4f8a-9b52-3ee134342b41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:02:53.350351 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41773549-c7e2-4f8a-9b52-3ee134342b41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:53.350620 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41773549-c7e2-4f8a-9b52-3ee134342b41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:53.351282 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41773549-c7e2-4f8a-9b52-3ee134342b41 admin admin] PATH_INFO: `/v3/users/1db1808db098471facffc352c3a7e52c` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:53.352142 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41773549-c7e2-4f8a-9b52-3ee134342b41 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1db1808db098471facffc352c3a7e52c)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:02:53.354066 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41773549-c7e2-4f8a-9b52-3ee134342b41 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:02:53.393589 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8756e497-2d41-4359-a8da-2e668d86348a admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:02:53.398340 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b49ff55-f817-448e-b5f7-e1addaef6c0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:02:53.399855 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b49ff55-f817-448e-b5f7-e1addaef6c0e admin admin] REQUEST_METHOD: `PUT` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:53.400392 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b49ff55-f817-448e-b5f7-e1addaef6c0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:53.400670 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b49ff55-f817-448e-b5f7-e1addaef6c0e admin admin] PATH_INFO: `/v3/projects/4a7333976980411f9b3db645d21cba20/users/1d68afdb08c042c1b9ac15318f34650c/roles/e2a5bbf3200a4d4091d72bc7ff78c34d` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:53.404610 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-41773549-c7e2-4f8a-9b52-3ee134342b41 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1db1808db098471facffc352c3a7e52c'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:53.405329 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-41773549-c7e2-4f8a-9b52-3ee134342b41 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1db1808db098471facffc352c3a7e52c'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:53.406713 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:02:53.414941 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-41773549-c7e2-4f8a-9b52-3ee134342b41 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1db1808db098471facffc352c3a7e52c'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:53.416705 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b49ff55-f817-448e-b5f7-e1addaef6c0e admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a7333976980411f9b3db645d21cba20, user_id=1d68afdb08c042c1b9ac15318f34650c, role_id=e2a5bbf3200a4d4091d72bc7ff78c34d)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:02:53.419461 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b49ff55-f817-448e-b5f7-e1addaef6c0e admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:02:53.423025 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-41773549-c7e2-4f8a-9b52-3ee134342b41 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1db1808db098471facffc352c3a7e52c'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:53.423688 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-41773549-c7e2-4f8a-9b52-3ee134342b41 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1db1808db098471facffc352c3a7e52c'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:53.433640 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1113/2195] 10.222.0.46 () {60 vars in 1482 bytes} [Thu Jun 23 16:02:53 2022] PUT /identity/v3/projects/4a7333976980411f9b3db645d21cba20/users/1d68afdb08c042c1b9ac15318f34650c/roles/e2a5bbf3200a4d4091d72bc7ff78c34d => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 16:02:53.434579 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b49ff55-f817-448e-b5f7-e1addaef6c0e admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:02:53.436277 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-4b49ff55-f817-448e-b5f7-e1addaef6c0e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:02:53.450333 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1083/2196] 10.222.0.46 () {58 vars in 1223 bytes} [Thu Jun 23 16:02:53 2022] DELETE /identity/v3/users/1db1808db098471facffc352c3a7e52c => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 16:02:53.451680 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d606739-a5fe-4a4e-b8d2-a1adc4500293 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:53.452148 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d606739-a5fe-4a4e-b8d2-a1adc4500293 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:53.452442 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d606739-a5fe-4a4e-b8d2-a1adc4500293 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:53.456901 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:02:53.481517 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-9d606739-a5fe-4a4e-b8d2-a1adc4500293 None None] MFA Rules not processed for user `1d68afdb08c042c1b9ac15318f34650c`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:02:53.482834 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-9d606739-a5fe-4a4e-b8d2-a1adc4500293 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:02:53.506640 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b49ff55-f817-448e-b5f7-e1addaef6c0e admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:02:53.560931 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1084/2197] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:02:53 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 16:02:53.565579 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0409f538-4a38-4736-8b8c-173cac918670 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:53.565579 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0409f538-4a38-4736-8b8c-173cac918670 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:53.565579 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0409f538-4a38-4736-8b8c-173cac918670 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:53.577103 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b59691-0afc-4baf-b07f-fc1ea23520bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:02:53.577103 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b59691-0afc-4baf-b07f-fc1ea23520bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:53.577103 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b59691-0afc-4baf-b07f-fc1ea23520bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:53.577469 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b59691-0afc-4baf-b07f-fc1ea23520bd admin admin] PATH_INFO: `/v3/users/c6642b7bdc5e4804aa0923605872079c` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:53.578598 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b59691-0afc-4baf-b07f-fc1ea23520bd admin admin] RBAC: Authorizing `identity:delete_user(user_id=c6642b7bdc5e4804aa0923605872079c)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:02:53.580711 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b59691-0afc-4baf-b07f-fc1ea23520bd admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:02:53.586570 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-0409f538-4a38-4736-8b8c-173cac918670 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:02:53.595880 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-0409f538-4a38-4736-8b8c-173cac918670 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:02:53.606590 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1085/2198] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:02:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 16:02:53.607666 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0409f538-4a38-4736-8b8c-173cac918670 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:02:53.610677 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-0409f538-4a38-4736-8b8c-173cac918670 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:02:53.664859 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-83b59691-0afc-4baf-b07f-fc1ea23520bd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c6642b7bdc5e4804aa0923605872079c'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:53.676153 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-83b59691-0afc-4baf-b07f-fc1ea23520bd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c6642b7bdc5e4804aa0923605872079c'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:53.676902 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-83b59691-0afc-4baf-b07f-fc1ea23520bd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c6642b7bdc5e4804aa0923605872079c'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:53.677578 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-83b59691-0afc-4baf-b07f-fc1ea23520bd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c6642b7bdc5e4804aa0923605872079c'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:53.694072 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-83b59691-0afc-4baf-b07f-fc1ea23520bd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c6642b7bdc5e4804aa0923605872079c'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:53.702770 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0409f538-4a38-4736-8b8c-173cac918670 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:02:53.714454 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207bb7fb-57f8-4f32-93ad-77254cb6a7dc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce18eaaba3a6477684fff1fd98ec8b51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:02:53.715455 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207bb7fb-57f8-4f32-93ad-77254cb6a7dc service cinder] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:53.716189 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207bb7fb-57f8-4f32-93ad-77254cb6a7dc service cinder] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:53.716621 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207bb7fb-57f8-4f32-93ad-77254cb6a7dc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:53.718084 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1114/2199] 10.222.0.46 () {58 vars in 1223 bytes} [Thu Jun 23 16:02:53 2022] DELETE /identity/v3/users/c6642b7bdc5e4804aa0923605872079c => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 16:02:53.719242 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-207bb7fb-57f8-4f32-93ad-77254cb6a7dc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:02:53.719388 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b59691-0afc-4baf-b07f-fc1ea23520bd admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:02:53.720874 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-83b59691-0afc-4baf-b07f-fc1ea23520bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:02:53.733855 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207bb7fb-57f8-4f32-93ad-77254cb6a7dc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:02:53.735502 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207bb7fb-57f8-4f32-93ad-77254cb6a7dc service cinder] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:02:53.748004 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1086/2200] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jun 23 16:02:53 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 16:02:53.765284 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207bb7fb-57f8-4f32-93ad-77254cb6a7dc service cinder] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:02:53.769909 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-207bb7fb-57f8-4f32-93ad-77254cb6a7dc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:02:53.800884 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:02:53.801334 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207bb7fb-57f8-4f32-93ad-77254cb6a7dc service cinder] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:02:53.816662 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b59691-0afc-4baf-b07f-fc1ea23520bd admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:02:53.825467 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cda470f-f08b-44a9-8e44-57df5093c8f1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51784064951a4619ae09b9167fb0fd0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:02:53.826415 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cda470f-f08b-44a9-8e44-57df5093c8f1 service neutron] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:53.827115 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cda470f-f08b-44a9-8e44-57df5093c8f1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:53.827489 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cda470f-f08b-44a9-8e44-57df5093c8f1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:53.829648 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-8cda470f-f08b-44a9-8e44-57df5093c8f1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:02:53.872482 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:02:53.885004 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11cf3319-644b-4374-818d-7a8552eba73a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:02:53.886827 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cf3319-644b-4374-818d-7a8552eba73a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:53.887373 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cf3319-644b-4374-818d-7a8552eba73a admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:53.887792 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cf3319-644b-4374-818d-7a8552eba73a admin admin] PATH_INFO: `/v3/users/1d68afdb08c042c1b9ac15318f34650c` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:53.889191 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11cf3319-644b-4374-818d-7a8552eba73a admin admin] RBAC: Authorizing `identity:delete_user(user_id=1d68afdb08c042c1b9ac15318f34650c)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:02:53.892520 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11cf3319-644b-4374-818d-7a8552eba73a admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:02:53.906814 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cda470f-f08b-44a9-8e44-57df5093c8f1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:02:53.908374 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cda470f-f08b-44a9-8e44-57df5093c8f1 service neutron] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:02:53.921131 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1115/2201] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jun 23 16:02:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 16:02:53.955561 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-11cf3319-644b-4374-818d-7a8552eba73a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1d68afdb08c042c1b9ac15318f34650c'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:53.956417 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-11cf3319-644b-4374-818d-7a8552eba73a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1d68afdb08c042c1b9ac15318f34650c'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:53.966544 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-11cf3319-644b-4374-818d-7a8552eba73a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1d68afdb08c042c1b9ac15318f34650c'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:53.990012 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-11cf3319-644b-4374-818d-7a8552eba73a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1d68afdb08c042c1b9ac15318f34650c'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:53.991554 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-11cf3319-644b-4374-818d-7a8552eba73a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1d68afdb08c042c1b9ac15318f34650c'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:54.019175 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1087/2202] 10.222.0.46 () {58 vars in 1223 bytes} [Thu Jun 23 16:02:53 2022] DELETE /identity/v3/users/1d68afdb08c042c1b9ac15318f34650c => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 16:02:54.029242 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc4579c-1cb8-4e4b-9098-1057359e9860 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:54.029242 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc4579c-1cb8-4e4b-9098-1057359e9860 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:54.029242 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc4579c-1cb8-4e4b-9098-1057359e9860 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:54.046247 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-1cc4579c-1cb8-4e4b-9098-1057359e9860 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:02:54.047765 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-1cc4579c-1cb8-4e4b-9098-1057359e9860 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:02:54.153987 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1116/2203] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:02:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 16:02:54.168237 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11cf3319-644b-4374-818d-7a8552eba73a admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:02:54.169697 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-11cf3319-644b-4374-818d-7a8552eba73a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:02:54.239434 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11cf3319-644b-4374-818d-7a8552eba73a admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:02:54.243791 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f06eb6-924f-463a-902a-22a5045aff46 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51784064951a4619ae09b9167fb0fd0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:02:54.244651 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f06eb6-924f-463a-902a-22a5045aff46 service neutron] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:54.244939 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f06eb6-924f-463a-902a-22a5045aff46 service neutron] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:54.245214 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f06eb6-924f-463a-902a-22a5045aff46 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:54.264819 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f06eb6-924f-463a-902a-22a5045aff46 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:02:54.266497 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f06eb6-924f-463a-902a-22a5045aff46 service neutron] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:02:54.279810 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1088/2204] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jun 23 16:02:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 16:02:54.471112 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc4579c-1cb8-4e4b-9098-1057359e9860 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:02:54.473423 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-1cc4579c-1cb8-4e4b-9098-1057359e9860 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:02:54.488727 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc4579c-1cb8-4e4b-9098-1057359e9860 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:02:54.495103 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0701acf4-9ff0-4874-8901-b6801b676dff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:02:54.496299 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0701acf4-9ff0-4874-8901-b6801b676dff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:54.497001 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0701acf4-9ff0-4874-8901-b6801b676dff admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:54.497521 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0701acf4-9ff0-4874-8901-b6801b676dff admin admin] PATH_INFO: `/v3/projects/bc64beda3ae44fa9ba126c1eabdd536f` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:54.499003 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0701acf4-9ff0-4874-8901-b6801b676dff admin admin] RBAC: Authorizing `identity:delete_project(project_id=bc64beda3ae44fa9ba126c1eabdd536f)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:02:54.501993 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0701acf4-9ff0-4874-8901-b6801b676dff admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:02:54.608648 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-0701acf4-9ff0-4874-8901-b6801b676dff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bc64beda3ae44fa9ba126c1eabdd536f', 'request_id': 'req-0701acf4-9ff0-4874-8901-b6801b676dff'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:54.616572 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-0701acf4-9ff0-4874-8901-b6801b676dff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bc64beda3ae44fa9ba126c1eabdd536f', 'request_id': 'req-0701acf4-9ff0-4874-8901-b6801b676dff'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:54.625183 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-0701acf4-9ff0-4874-8901-b6801b676dff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bc64beda3ae44fa9ba126c1eabdd536f', 'request_id': 'req-0701acf4-9ff0-4874-8901-b6801b676dff'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:54.641479 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-0701acf4-9ff0-4874-8901-b6801b676dff admin admin] The token cache is being invalidate because project bc64beda3ae44fa9ba126c1eabdd536f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80153) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 16:02:54.641479 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-0701acf4-9ff0-4874-8901-b6801b676dff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:54.642235 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-0701acf4-9ff0-4874-8901-b6801b676dff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:54.644071 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1117/2205] 10.222.0.46 () {58 vars in 1232 bytes} [Thu Jun 23 16:02:54 2022] DELETE /identity/v3/projects/bc64beda3ae44fa9ba126c1eabdd536f => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 16:02:54.654532 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f06eb6-924f-463a-902a-22a5045aff46 service neutron] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:02:54.669126 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f06eb6-924f-463a-902a-22a5045aff46 service neutron] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:02:54.682034 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0701acf4-9ff0-4874-8901-b6801b676dff admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:02:54.683487 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-0701acf4-9ff0-4874-8901-b6801b676dff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:02:54.719698 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:02:54.753291 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcdb807-2fec-4b0f-a863-5ae30a20abf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:02:54.754191 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcdb807-2fec-4b0f-a863-5ae30a20abf9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:54.754476 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcdb807-2fec-4b0f-a863-5ae30a20abf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:54.754786 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcdb807-2fec-4b0f-a863-5ae30a20abf9 admin admin] PATH_INFO: `/v3/users/3487f733ec204c5db889531c181bb657` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:54.756020 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcdb807-2fec-4b0f-a863-5ae30a20abf9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3487f733ec204c5db889531c181bb657)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:02:54.758541 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcdb807-2fec-4b0f-a863-5ae30a20abf9 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:02:54.781545 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0701acf4-9ff0-4874-8901-b6801b676dff admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:02:54.782981 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:02:54.797133 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80212e5c-afd1-4bec-ba34-f1ed66c415ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:02:54.798062 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80212e5c-afd1-4bec-ba34-f1ed66c415ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:54.798609 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80212e5c-afd1-4bec-ba34-f1ed66c415ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:54.798820 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80212e5c-afd1-4bec-ba34-f1ed66c415ca admin admin] PATH_INFO: `/v3/projects/4a7333976980411f9b3db645d21cba20` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:54.800324 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80212e5c-afd1-4bec-ba34-f1ed66c415ca admin admin] RBAC: Authorizing `identity:delete_project(project_id=4a7333976980411f9b3db645d21cba20)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:02:54.802276 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80212e5c-afd1-4bec-ba34-f1ed66c415ca admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:02:54.837128 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-9dcdb807-2fec-4b0f-a863-5ae30a20abf9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3487f733ec204c5db889531c181bb657'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:54.838281 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-9dcdb807-2fec-4b0f-a863-5ae30a20abf9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3487f733ec204c5db889531c181bb657'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:54.849362 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-9dcdb807-2fec-4b0f-a863-5ae30a20abf9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3487f733ec204c5db889531c181bb657'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:54.861089 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-9dcdb807-2fec-4b0f-a863-5ae30a20abf9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3487f733ec204c5db889531c181bb657'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:54.862100 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-9dcdb807-2fec-4b0f-a863-5ae30a20abf9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3487f733ec204c5db889531c181bb657'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:54.866208 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-80212e5c-afd1-4bec-ba34-f1ed66c415ca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4a7333976980411f9b3db645d21cba20', 'request_id': 'req-80212e5c-afd1-4bec-ba34-f1ed66c415ca'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:54.872732 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-80212e5c-afd1-4bec-ba34-f1ed66c415ca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4a7333976980411f9b3db645d21cba20', 'request_id': 'req-80212e5c-afd1-4bec-ba34-f1ed66c415ca'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:54.880562 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-80212e5c-afd1-4bec-ba34-f1ed66c415ca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4a7333976980411f9b3db645d21cba20', 'request_id': 'req-80212e5c-afd1-4bec-ba34-f1ed66c415ca'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:54.888325 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1089/2206] 10.222.0.46 () {58 vars in 1223 bytes} [Thu Jun 23 16:02:54 2022] DELETE /identity/v3/users/3487f733ec204c5db889531c181bb657 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 16:02:54.891924 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcdb807-2fec-4b0f-a863-5ae30a20abf9 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:02:54.894989 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-9dcdb807-2fec-4b0f-a863-5ae30a20abf9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:02:54.911200 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-80212e5c-afd1-4bec-ba34-f1ed66c415ca admin admin] The token cache is being invalidate because project 4a7333976980411f9b3db645d21cba20 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80153) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 16:02:54.911590 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-80212e5c-afd1-4bec-ba34-f1ed66c415ca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:54.912175 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-80212e5c-afd1-4bec-ba34-f1ed66c415ca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:54.921104 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1118/2207] 10.222.0.46 () {58 vars in 1232 bytes} [Thu Jun 23 16:02:54 2022] DELETE /identity/v3/projects/4a7333976980411f9b3db645d21cba20 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 16:02:54.923627 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c175ab-a66b-403b-9a59-fa743f85e086 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:54.924715 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c175ab-a66b-403b-9a59-fa743f85e086 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:54.925169 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c175ab-a66b-403b-9a59-fa743f85e086 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:54.956756 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-88c175ab-a66b-403b-9a59-fa743f85e086 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:02:54.957979 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-88c175ab-a66b-403b-9a59-fa743f85e086 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:02:54.987919 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcdb807-2fec-4b0f-a863-5ae30a20abf9 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:02:54.993544 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f3aae5-8333-4557-a268-e86a9b58a749 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:02:54.995543 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f3aae5-8333-4557-a268-e86a9b58a749 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:54.996005 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f3aae5-8333-4557-a268-e86a9b58a749 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:54.996385 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f3aae5-8333-4557-a268-e86a9b58a749 admin admin] PATH_INFO: `/v3/projects/59bbb0eed10847dda49906f85353f668` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:54.998090 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f3aae5-8333-4557-a268-e86a9b58a749 admin admin] RBAC: Authorizing `identity:delete_project(project_id=59bbb0eed10847dda49906f85353f668)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:02:55.000382 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f3aae5-8333-4557-a268-e86a9b58a749 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:02:55.063160 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1119/2208] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:02:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 16:02:55.065662 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374df02e-dd7b-49b2-9542-637193190c88 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:55.065977 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374df02e-dd7b-49b2-9542-637193190c88 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:55.066251 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374df02e-dd7b-49b2-9542-637193190c88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:55.090253 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-374df02e-dd7b-49b2-9542-637193190c88 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:02:55.091947 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-374df02e-dd7b-49b2-9542-637193190c88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:02:55.100100 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-77f3aae5-8333-4557-a268-e86a9b58a749 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '59bbb0eed10847dda49906f85353f668', 'request_id': 'req-77f3aae5-8333-4557-a268-e86a9b58a749'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:55.108460 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-77f3aae5-8333-4557-a268-e86a9b58a749 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '59bbb0eed10847dda49906f85353f668', 'request_id': 'req-77f3aae5-8333-4557-a268-e86a9b58a749'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:55.124931 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-77f3aae5-8333-4557-a268-e86a9b58a749 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '59bbb0eed10847dda49906f85353f668', 'request_id': 'req-77f3aae5-8333-4557-a268-e86a9b58a749'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:55.155305 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-77f3aae5-8333-4557-a268-e86a9b58a749 admin admin] The token cache is being invalidate because project 59bbb0eed10847dda49906f85353f668 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80154) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 16:02:55.155676 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-77f3aae5-8333-4557-a268-e86a9b58a749 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:55.156431 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-77f3aae5-8333-4557-a268-e86a9b58a749 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:55.160777 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1090/2209] 10.222.0.46 () {58 vars in 1232 bytes} [Thu Jun 23 16:02:54 2022] DELETE /identity/v3/projects/59bbb0eed10847dda49906f85353f668 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 16:02:55.166773 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ce252b-95fd-4c28-820c-235a01f2f381 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:55.167090 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ce252b-95fd-4c28-820c-235a01f2f381 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:55.167385 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ce252b-95fd-4c28-820c-235a01f2f381 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:55.213274 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:02:55.234944 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1120/2210] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:02:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 16:02:55.240448 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374df02e-dd7b-49b2-9542-637193190c88 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:02:55.246386 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-83ce252b-95fd-4c28-820c-235a01f2f381 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:02:55.248345 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-83ce252b-95fd-4c28-820c-235a01f2f381 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:02:55.265415 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374df02e-dd7b-49b2-9542-637193190c88 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:02:55.266761 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1091/2211] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:02:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 16:02:55.272275 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a68889-fca3-4cfb-af8c-6b2fdfd2e2e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:02:55.273195 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a68889-fca3-4cfb-af8c-6b2fdfd2e2e2 admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:55.273484 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a68889-fca3-4cfb-af8c-6b2fdfd2e2e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:55.274133 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a68889-fca3-4cfb-af8c-6b2fdfd2e2e2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:55.274968 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a68889-fca3-4cfb-af8c-6b2fdfd2e2e2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 16:02:55.275671 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a68889-fca3-4cfb-af8c-6b2fdfd2e2e2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:02:55.276334 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82239231-1e84-4da5-9370-bb5a683a0ad2 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:55.276637 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82239231-1e84-4da5-9370-bb5a683a0ad2 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:55.277425 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82239231-1e84-4da5-9370-bb5a683a0ad2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:55.277536 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a68889-fca3-4cfb-af8c-6b2fdfd2e2e2 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:02:55.291392 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1121/2212] 10.222.0.46 () {58 vars in 1152 bytes} [Thu Jun 23 16:02:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 16:02:55.295585 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a68889-fca3-4cfb-af8c-6b2fdfd2e2e2 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:02:55.308895 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a68889-fca3-4cfb-af8c-6b2fdfd2e2e2 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:02:55.315553 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fce3de-82f6-4525-bb66-437350a1df39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:02:55.316542 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fce3de-82f6-4525-bb66-437350a1df39 admin admin] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:55.317206 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fce3de-82f6-4525-bb66-437350a1df39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:55.318489 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fce3de-82f6-4525-bb66-437350a1df39 admin admin] PATH_INFO: `/v3/projects` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:55.319602 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fce3de-82f6-4525-bb66-437350a1df39 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:02:55.323413 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-82239231-1e84-4da5-9370-bb5a683a0ad2 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:02:55.324794 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-82239231-1e84-4da5-9370-bb5a683a0ad2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:02:55.325786 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fce3de-82f6-4525-bb66-437350a1df39 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:02:55.336126 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1092/2213] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:02:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 16:02:55.340714 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82239231-1e84-4da5-9370-bb5a683a0ad2 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:02:55.353012 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82239231-1e84-4da5-9370-bb5a683a0ad2 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:02:55.358765 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd4bd0a-a067-4ccc-a9b1-60b51e51fcf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:02:55.359785 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd4bd0a-a067-4ccc-a9b1-60b51e51fcf1 admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:55.360204 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd4bd0a-a067-4ccc-a9b1-60b51e51fcf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:55.360613 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd4bd0a-a067-4ccc-a9b1-60b51e51fcf1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:55.361543 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd4bd0a-a067-4ccc-a9b1-60b51e51fcf1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 16:02:55.361951 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd4bd0a-a067-4ccc-a9b1-60b51e51fcf1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:02:55.363904 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd4bd0a-a067-4ccc-a9b1-60b51e51fcf1 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:02:55.368259 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:02:55.378268 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1122/2214] 10.222.0.46 () {60 vars in 1152 bytes} [Thu Jun 23 16:02:55 2022] POST /identity/v3/projects => generated 373 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 16:02:55.378790 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fce3de-82f6-4525-bb66-437350a1df39 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:02:55.390447 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1093/2215] 10.222.0.46 () {58 vars in 1152 bytes} [Thu Jun 23 16:02:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 16:02:55.392088 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd4bd0a-a067-4ccc-a9b1-60b51e51fcf1 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:02:55.407497 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd4bd0a-a067-4ccc-a9b1-60b51e51fcf1 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:02:55.482226 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fce3de-82f6-4525-bb66-437350a1df39 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:02:55.487882 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64bafe9-339d-488d-8a7b-0d00eacb8b26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:02:55.488582 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64bafe9-339d-488d-8a7b-0d00eacb8b26 admin admin] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:55.488880 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64bafe9-339d-488d-8a7b-0d00eacb8b26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:55.489158 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64bafe9-339d-488d-8a7b-0d00eacb8b26 admin admin] PATH_INFO: `/v3/users` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:55.490155 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64bafe9-339d-488d-8a7b-0d00eacb8b26 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:02:55.492998 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64bafe9-339d-488d-8a7b-0d00eacb8b26 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:02:55.495369 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99986511-77d8-4e49-90ad-507bd7efe5aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:02:55.496069 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99986511-77d8-4e49-90ad-507bd7efe5aa admin admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:55.496357 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99986511-77d8-4e49-90ad-507bd7efe5aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:55.496643 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99986511-77d8-4e49-90ad-507bd7efe5aa admin admin] PATH_INFO: `/v3/projects` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:55.497408 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99986511-77d8-4e49-90ad-507bd7efe5aa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:02:55.499501 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99986511-77d8-4e49-90ad-507bd7efe5aa admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:02:55.531802 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1123/2216] 10.222.0.46 () {60 vars in 1143 bytes} [Thu Jun 23 16:02:55 2022] POST /identity/v3/users => generated 287 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 16:02:55.544366 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64bafe9-339d-488d-8a7b-0d00eacb8b26 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:02:55.556224 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1094/2217] 10.222.0.46 () {60 vars in 1152 bytes} [Thu Jun 23 16:02:55 2022] POST /identity/v3/projects => generated 371 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 16:02:55.564441 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99986511-77d8-4e49-90ad-507bd7efe5aa admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:02:55.576576 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64bafe9-339d-488d-8a7b-0d00eacb8b26 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:02:55.677478 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba3bbbf-89d4-4571-98ba-ceb9483f11c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:02:55.678171 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba3bbbf-89d4-4571-98ba-ceb9483f11c2 admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:55.678445 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba3bbbf-89d4-4571-98ba-ceb9483f11c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:55.678751 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba3bbbf-89d4-4571-98ba-ceb9483f11c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:55.685446 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba3bbbf-89d4-4571-98ba-ceb9483f11c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:02:55.687055 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99986511-77d8-4e49-90ad-507bd7efe5aa admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:02:55.687398 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba3bbbf-89d4-4571-98ba-ceb9483f11c2 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:02:55.693992 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9aee2ec-a286-4a81-a3ad-2dd68e6a6319 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:02:55.695065 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9aee2ec-a286-4a81-a3ad-2dd68e6a6319 admin admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:55.696402 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9aee2ec-a286-4a81-a3ad-2dd68e6a6319 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:55.697223 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9aee2ec-a286-4a81-a3ad-2dd68e6a6319 admin admin] PATH_INFO: `/v3/users` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:55.698412 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9aee2ec-a286-4a81-a3ad-2dd68e6a6319 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:02:55.704065 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1124/2218] 10.222.0.46 () {58 vars in 1121 bytes} [Thu Jun 23 16:02:55 2022] GET /identity/v3/roles => generated 1420 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 16:02:55.704838 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba3bbbf-89d4-4571-98ba-ceb9483f11c2 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:02:55.705237 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9aee2ec-a286-4a81-a3ad-2dd68e6a6319 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:02:55.710344 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-6ba3bbbf-89d4-4571-98ba-ceb9483f11c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:02:55.744809 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba3bbbf-89d4-4571-98ba-ceb9483f11c2 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:02:55.752250 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5325e364-41b0-44cb-a403-10b93ada8b9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:02:55.753772 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5325e364-41b0-44cb-a403-10b93ada8b9b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:55.754134 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5325e364-41b0-44cb-a403-10b93ada8b9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:55.754329 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5325e364-41b0-44cb-a403-10b93ada8b9b admin admin] PATH_INFO: `/v3/projects/553c97c8e1044a318fe94a832a7c9cb9` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:55.763208 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5325e364-41b0-44cb-a403-10b93ada8b9b admin admin] RBAC: Authorizing `identity:delete_project(project_id=553c97c8e1044a318fe94a832a7c9cb9)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:02:55.765241 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5325e364-41b0-44cb-a403-10b93ada8b9b admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:02:55.767504 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1095/2219] 10.222.0.46 () {60 vars in 1143 bytes} [Thu Jun 23 16:02:55 2022] POST /identity/v3/users => generated 286 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 16:02:55.769082 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9aee2ec-a286-4a81-a3ad-2dd68e6a6319 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:02:55.781455 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9aee2ec-a286-4a81-a3ad-2dd68e6a6319 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:02:55.793064 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f867ca-d1b3-4263-96cd-0001e9271381 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:02:55.793917 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f867ca-d1b3-4263-96cd-0001e9271381 admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:55.795081 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f867ca-d1b3-4263-96cd-0001e9271381 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:55.795402 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f867ca-d1b3-4263-96cd-0001e9271381 admin admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:55.796428 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f867ca-d1b3-4263-96cd-0001e9271381 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:02:55.798770 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f867ca-d1b3-4263-96cd-0001e9271381 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:02:55.811976 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1096/2220] 10.222.0.46 () {58 vars in 1121 bytes} [Thu Jun 23 16:02:55 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 16:02:55.813121 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f867ca-d1b3-4263-96cd-0001e9271381 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:02:55.819469 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=94,keystone:INSERT=4,keystone:DELETE=2 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:02:55.824818 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f867ca-d1b3-4263-96cd-0001e9271381 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:02:55.834725 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-5325e364-41b0-44cb-a403-10b93ada8b9b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '553c97c8e1044a318fe94a832a7c9cb9', 'request_id': 'req-5325e364-41b0-44cb-a403-10b93ada8b9b'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:55.841237 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-5325e364-41b0-44cb-a403-10b93ada8b9b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '553c97c8e1044a318fe94a832a7c9cb9', 'request_id': 'req-5325e364-41b0-44cb-a403-10b93ada8b9b'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:55.850192 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-5325e364-41b0-44cb-a403-10b93ada8b9b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '553c97c8e1044a318fe94a832a7c9cb9', 'request_id': 'req-5325e364-41b0-44cb-a403-10b93ada8b9b'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:55.875350 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-5325e364-41b0-44cb-a403-10b93ada8b9b admin admin] The token cache is being invalidate because project 553c97c8e1044a318fe94a832a7c9cb9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80153) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 16:02:55.875658 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-5325e364-41b0-44cb-a403-10b93ada8b9b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:55.876408 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-5325e364-41b0-44cb-a403-10b93ada8b9b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:55.881666 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1125/2221] 10.222.0.46 () {58 vars in 1232 bytes} [Thu Jun 23 16:02:55 2022] DELETE /identity/v3/projects/553c97c8e1044a318fe94a832a7c9cb9 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 16:02:55.882837 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5325e364-41b0-44cb-a403-10b93ada8b9b admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:02:55.884739 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-5325e364-41b0-44cb-a403-10b93ada8b9b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:02:55.889226 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b81d18-3b57-4d38-ac02-8db4419c819c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:02:55.889997 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b81d18-3b57-4d38-ac02-8db4419c819c admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:55.890287 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b81d18-3b57-4d38-ac02-8db4419c819c admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:55.890567 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b81d18-3b57-4d38-ac02-8db4419c819c admin admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:55.891411 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b81d18-3b57-4d38-ac02-8db4419c819c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:02:55.893146 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b81d18-3b57-4d38-ac02-8db4419c819c admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:02:55.894665 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:02:55.898232 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5325e364-41b0-44cb-a403-10b93ada8b9b admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:02:55.905037 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4867f5c1-e6ba-4658-afef-74fa73679d61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:02:55.906828 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4867f5c1-e6ba-4658-afef-74fa73679d61 admin admin] REQUEST_METHOD: `PUT` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:55.907110 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4867f5c1-e6ba-4658-afef-74fa73679d61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:55.907394 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4867f5c1-e6ba-4658-afef-74fa73679d61 admin admin] PATH_INFO: `/v3/projects/bbde90b595244aef82fd4d52d4abfaa8/users/2399b4fc64f74475bd8dc7b1634a2c2a/roles/e2a5bbf3200a4d4091d72bc7ff78c34d` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:55.911986 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1097/2222] 10.222.0.46 () {58 vars in 1121 bytes} [Thu Jun 23 16:02:55 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 16:02:55.913428 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da2e622-424a-40ba-8512-074648fbe873 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:55.913708 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da2e622-424a-40ba-8512-074648fbe873 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:55.913995 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da2e622-424a-40ba-8512-074648fbe873 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:55.929765 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4867f5c1-e6ba-4658-afef-74fa73679d61 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bbde90b595244aef82fd4d52d4abfaa8, user_id=2399b4fc64f74475bd8dc7b1634a2c2a, role_id=e2a5bbf3200a4d4091d72bc7ff78c34d)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:02:55.932982 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4867f5c1-e6ba-4658-afef-74fa73679d61 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:02:55.933876 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-1da2e622-424a-40ba-8512-074648fbe873 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:02:55.935150 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-1da2e622-424a-40ba-8512-074648fbe873 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:02:55.944403 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1098/2223] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:02:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 16:02:55.945563 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da2e622-424a-40ba-8512-074648fbe873 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:02:55.947192 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-1da2e622-424a-40ba-8512-074648fbe873 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:02:55.953463 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1126/2224] 10.222.0.46 () {60 vars in 1482 bytes} [Thu Jun 23 16:02:55 2022] PUT /identity/v3/projects/bbde90b595244aef82fd4d52d4abfaa8/users/2399b4fc64f74475bd8dc7b1634a2c2a/roles/e2a5bbf3200a4d4091d72bc7ff78c34d => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 16:02:55.954870 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d07f01-1afb-4831-a8fa-353e81265bb6 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:55.955161 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d07f01-1afb-4831-a8fa-353e81265bb6 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:55.955439 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d07f01-1afb-4831-a8fa-353e81265bb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:55.963234 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da2e622-424a-40ba-8512-074648fbe873 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:02:55.974988 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-d9d07f01-1afb-4831-a8fa-353e81265bb6 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:02:55.976227 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-d9d07f01-1afb-4831-a8fa-353e81265bb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:02:56.040715 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1127/2225] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:02:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 16:02:56.044012 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f752dd-ead1-48d3-bd03-5a1282e153a6 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:56.044126 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b530956-356a-4e00-b2da-fb081750afb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:02:56.044245 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f752dd-ead1-48d3-bd03-5a1282e153a6 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:56.044245 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f752dd-ead1-48d3-bd03-5a1282e153a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:56.044427 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b530956-356a-4e00-b2da-fb081750afb0 admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:56.044427 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b530956-356a-4e00-b2da-fb081750afb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:56.044583 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b530956-356a-4e00-b2da-fb081750afb0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:56.045402 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b530956-356a-4e00-b2da-fb081750afb0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:02:56.047224 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b530956-356a-4e00-b2da-fb081750afb0 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:02:56.060581 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1099/2226] 10.222.0.46 () {58 vars in 1121 bytes} [Thu Jun 23 16:02:55 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 16:02:56.061445 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b530956-356a-4e00-b2da-fb081750afb0 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:02:56.063396 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-35f752dd-ead1-48d3-bd03-5a1282e153a6 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:02:56.064951 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-35f752dd-ead1-48d3-bd03-5a1282e153a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:02:56.074725 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1128/2227] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:02:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 16:02:56.075688 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f752dd-ead1-48d3-bd03-5a1282e153a6 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:02:56.085900 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b530956-356a-4e00-b2da-fb081750afb0 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:02:56.089456 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f752dd-ead1-48d3-bd03-5a1282e153a6 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:02:56.091837 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf2aad4-0b11-46ce-ae31-eb645acb9684 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:02:56.093138 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf2aad4-0b11-46ce-ae31-eb645acb9684 admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:56.093614 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf2aad4-0b11-46ce-ae31-eb645acb9684 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:56.094250 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf2aad4-0b11-46ce-ae31-eb645acb9684 admin admin] PATH_INFO: `/v3/domains` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:56.095276 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf2aad4-0b11-46ce-ae31-eb645acb9684 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 16:02:56.095765 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf2aad4-0b11-46ce-ae31-eb645acb9684 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:02:56.096565 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f659cd0d-48b2-4430-b607-e665475cacb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:02:56.097465 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f659cd0d-48b2-4430-b607-e665475cacb9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:56.097745 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f659cd0d-48b2-4430-b607-e665475cacb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:56.098019 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f659cd0d-48b2-4430-b607-e665475cacb9 admin admin] PATH_INFO: `/v3/projects/42158893b22d49449502910b10869641/users/a23cff1016ed41909027d228608f26e9/roles/e2a5bbf3200a4d4091d72bc7ff78c34d` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:56.099171 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf2aad4-0b11-46ce-ae31-eb645acb9684 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:02:56.119210 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1100/2228] 10.222.0.46 () {58 vars in 1152 bytes} [Thu Jun 23 16:02:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 16:02:56.120510 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf2aad4-0b11-46ce-ae31-eb645acb9684 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:02:56.122716 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-acf2aad4-0b11-46ce-ae31-eb645acb9684 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:02:56.141572 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f659cd0d-48b2-4430-b607-e665475cacb9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=42158893b22d49449502910b10869641, user_id=a23cff1016ed41909027d228608f26e9, role_id=e2a5bbf3200a4d4091d72bc7ff78c34d)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:02:56.146802 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f659cd0d-48b2-4430-b607-e665475cacb9 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:02:56.163736 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1129/2229] 10.222.0.46 () {60 vars in 1482 bytes} [Thu Jun 23 16:02:56 2022] PUT /identity/v3/projects/42158893b22d49449502910b10869641/users/a23cff1016ed41909027d228608f26e9/roles/e2a5bbf3200a4d4091d72bc7ff78c34d => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 16:02:56.165400 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f659cd0d-48b2-4430-b607-e665475cacb9 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:02:56.229649 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf2aad4-0b11-46ce-ae31-eb645acb9684 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:02:56.268498 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f659cd0d-48b2-4430-b607-e665475cacb9 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:02:56.284032 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1648ff05-75c8-44a7-b75b-3d90bc21a721 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:02:56.284733 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1648ff05-75c8-44a7-b75b-3d90bc21a721 admin admin] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:56.285024 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1648ff05-75c8-44a7-b75b-3d90bc21a721 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:56.285308 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1648ff05-75c8-44a7-b75b-3d90bc21a721 admin admin] PATH_INFO: `/v3/projects` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:56.286731 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1648ff05-75c8-44a7-b75b-3d90bc21a721 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:02:56.288896 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1648ff05-75c8-44a7-b75b-3d90bc21a721 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:02:56.307913 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:DELETE=5,keystone:SELECT=92,keystone:INSERT=3 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:02:56.333792 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf80786f-d313-4639-a129-90b0cdff0dca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51784064951a4619ae09b9167fb0fd0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:02:56.334488 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf80786f-d313-4639-a129-90b0cdff0dca service neutron] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:56.334790 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf80786f-d313-4639-a129-90b0cdff0dca service neutron] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:56.335205 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf80786f-d313-4639-a129-90b0cdff0dca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:56.338597 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1130/2230] 10.222.0.46 () {60 vars in 1152 bytes} [Thu Jun 23 16:02:56 2022] POST /identity/v3/projects => generated 385 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 16:02:56.339921 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbf5a94-518f-4966-b697-adfb2624648a None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:56.340219 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbf5a94-518f-4966-b697-adfb2624648a None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:56.340504 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbf5a94-518f-4966-b697-adfb2624648a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:56.348338 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf80786f-d313-4639-a129-90b0cdff0dca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:02:56.349823 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf80786f-d313-4639-a129-90b0cdff0dca service neutron] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:02:56.364128 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1101/2231] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jun 23 16:02:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 244 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 16:02:56.364985 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf80786f-d313-4639-a129-90b0cdff0dca service neutron] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:02:56.374637 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-abbf5a94-518f-4966-b697-adfb2624648a None None] MFA Rules not processed for user `a23cff1016ed41909027d228608f26e9`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:02:56.374870 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf80786f-d313-4639-a129-90b0cdff0dca service neutron] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:02:56.375982 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-abbf5a94-518f-4966-b697-adfb2624648a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:02:56.414965 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=100 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:02:56.450322 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88a6de0-ac64-46e2-a1f7-e11e3b4f42cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:02:56.451214 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88a6de0-ac64-46e2-a1f7-e11e3b4f42cf admin admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:56.451636 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88a6de0-ac64-46e2-a1f7-e11e3b4f42cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:56.452018 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88a6de0-ac64-46e2-a1f7-e11e3b4f42cf admin admin] PATH_INFO: `/v3/users` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:56.453008 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88a6de0-ac64-46e2-a1f7-e11e3b4f42cf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:02:56.455989 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88a6de0-ac64-46e2-a1f7-e11e3b4f42cf admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:02:56.482343 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1131/2232] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:02:56 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 16:02:56.491056 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1102/2233] 10.222.0.46 () {60 vars in 1143 bytes} [Thu Jun 23 16:02:56 2022] POST /identity/v3/users => generated 293 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 16:02:56.492913 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbf5a94-518f-4966-b697-adfb2624648a None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:02:56.495669 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88a6de0-ac64-46e2-a1f7-e11e3b4f42cf admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:02:56.498111 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-e88a6de0-ac64-46e2-a1f7-e11e3b4f42cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:02:56.507164 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbf5a94-518f-4966-b697-adfb2624648a None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:02:56.515332 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfec1832-df0d-4a0d-be14-948e3b292b11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:02:56.516074 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfec1832-df0d-4a0d-be14-948e3b292b11 admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:56.516361 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfec1832-df0d-4a0d-be14-948e3b292b11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:56.516639 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfec1832-df0d-4a0d-be14-948e3b292b11 admin admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:56.517690 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfec1832-df0d-4a0d-be14-948e3b292b11 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:02:56.519434 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfec1832-df0d-4a0d-be14-948e3b292b11 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:02:56.538813 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1132/2234] 10.222.0.46 () {58 vars in 1121 bytes} [Thu Jun 23 16:02:56 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 16:02:56.540524 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfec1832-df0d-4a0d-be14-948e3b292b11 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:02:56.559002 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfec1832-df0d-4a0d-be14-948e3b292b11 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:02:56.575779 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a20ad4c-a845-4a48-896e-b5c76934a249 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:02:56.577488 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a20ad4c-a845-4a48-896e-b5c76934a249 admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:56.577941 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a20ad4c-a845-4a48-896e-b5c76934a249 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:56.578266 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a20ad4c-a845-4a48-896e-b5c76934a249 admin admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:56.579222 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a20ad4c-a845-4a48-896e-b5c76934a249 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:02:56.581027 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a20ad4c-a845-4a48-896e-b5c76934a249 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:02:56.598702 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88a6de0-ac64-46e2-a1f7-e11e3b4f42cf admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:02:56.599480 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1133/2235] 10.222.0.46 () {58 vars in 1121 bytes} [Thu Jun 23 16:02:56 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 16:02:56.600678 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a20ad4c-a845-4a48-896e-b5c76934a249 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:02:56.603310 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f428ff54-fc33-41e6-98e2-32fad0bb1bc6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce18eaaba3a6477684fff1fd98ec8b51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:02:56.604387 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f428ff54-fc33-41e6-98e2-32fad0bb1bc6 service cinder] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:56.604672 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f428ff54-fc33-41e6-98e2-32fad0bb1bc6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:56.604946 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f428ff54-fc33-41e6-98e2-32fad0bb1bc6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:56.616549 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a20ad4c-a845-4a48-896e-b5c76934a249 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:02:56.621467 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f428ff54-fc33-41e6-98e2-32fad0bb1bc6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:02:56.624137 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f428ff54-fc33-41e6-98e2-32fad0bb1bc6 service cinder] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:02:56.624754 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a542649-f8c8-4299-acbc-89b28a19e800 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:02:56.626466 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a542649-f8c8-4299-acbc-89b28a19e800 admin admin] REQUEST_METHOD: `PUT` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:56.627020 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a542649-f8c8-4299-acbc-89b28a19e800 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:56.627505 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a542649-f8c8-4299-acbc-89b28a19e800 admin admin] PATH_INFO: `/v3/projects/3c0491559ff54189be425d60c5c4f36c/users/c5626c4a29674aaea2469595002260ae/roles/e2a5bbf3200a4d4091d72bc7ff78c34d` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:56.643922 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1103/2236] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jun 23 16:02:56 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 16:02:56.652130 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a542649-f8c8-4299-acbc-89b28a19e800 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c0491559ff54189be425d60c5c4f36c, user_id=c5626c4a29674aaea2469595002260ae, role_id=e2a5bbf3200a4d4091d72bc7ff78c34d)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:02:56.657303 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a542649-f8c8-4299-acbc-89b28a19e800 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:02:56.679472 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f428ff54-fc33-41e6-98e2-32fad0bb1bc6 service cinder] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:02:56.715232 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1134/2237] 10.222.0.46 () {60 vars in 1482 bytes} [Thu Jun 23 16:02:56 2022] PUT /identity/v3/projects/3c0491559ff54189be425d60c5c4f36c/users/c5626c4a29674aaea2469595002260ae/roles/e2a5bbf3200a4d4091d72bc7ff78c34d => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 16:02:56.715337 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f428ff54-fc33-41e6-98e2-32fad0bb1bc6 service cinder] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:02:56.717231 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864a25bf-58fd-4dcf-a857-663cb42b0cf4 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:56.717614 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864a25bf-58fd-4dcf-a857-663cb42b0cf4 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:56.718048 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864a25bf-58fd-4dcf-a857-663cb42b0cf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:56.788186 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-864a25bf-58fd-4dcf-a857-663cb42b0cf4 None None] MFA Rules not processed for user `c5626c4a29674aaea2469595002260ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:02:56.789475 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-864a25bf-58fd-4dcf-a857-663cb42b0cf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:02:56.812219 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f98d806-7820-4d57-8a87-8a7f073e7e48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:02:56.814106 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f98d806-7820-4d57-8a87-8a7f073e7e48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:56.814627 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f98d806-7820-4d57-8a87-8a7f073e7e48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:56.815374 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f98d806-7820-4d57-8a87-8a7f073e7e48 admin admin] PATH_INFO: `/v3/users/a23cff1016ed41909027d228608f26e9` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:56.817566 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f98d806-7820-4d57-8a87-8a7f073e7e48 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a23cff1016ed41909027d228608f26e9)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:02:56.821534 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f98d806-7820-4d57-8a87-8a7f073e7e48 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:02:56.881092 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1135/2238] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:02:56 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 16:02:56.887607 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864a25bf-58fd-4dcf-a857-663cb42b0cf4 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:02:56.892978 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-3f98d806-7820-4d57-8a87-8a7f073e7e48 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a23cff1016ed41909027d228608f26e9'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:56.893668 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-3f98d806-7820-4d57-8a87-8a7f073e7e48 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a23cff1016ed41909027d228608f26e9'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:56.899503 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864a25bf-58fd-4dcf-a857-663cb42b0cf4 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:02:56.905438 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-3f98d806-7820-4d57-8a87-8a7f073e7e48 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a23cff1016ed41909027d228608f26e9'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:56.908805 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07431ab-94f4-4d3f-9b5b-630cb63992e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:02:56.910195 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07431ab-94f4-4d3f-9b5b-630cb63992e5 admin admin] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:56.911015 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07431ab-94f4-4d3f-9b5b-630cb63992e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:56.911885 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07431ab-94f4-4d3f-9b5b-630cb63992e5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:56.912889 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07431ab-94f4-4d3f-9b5b-630cb63992e5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:02:56.915823 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07431ab-94f4-4d3f-9b5b-630cb63992e5 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:02:56.918242 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-3f98d806-7820-4d57-8a87-8a7f073e7e48 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a23cff1016ed41909027d228608f26e9'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:56.919336 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-3f98d806-7820-4d57-8a87-8a7f073e7e48 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a23cff1016ed41909027d228608f26e9'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:56.922301 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:02:56.943877 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1104/2239] 10.222.0.46 () {58 vars in 1223 bytes} [Thu Jun 23 16:02:56 2022] DELETE /identity/v3/users/a23cff1016ed41909027d228608f26e9 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 16:02:56.948654 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d561a1-d33f-4356-93b3-9f5cf474d2d9 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:56.948950 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d561a1-d33f-4356-93b3-9f5cf474d2d9 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:56.949757 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d561a1-d33f-4356-93b3-9f5cf474d2d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:56.951082 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1136/2240] 10.222.0.46 () {60 vars in 1152 bytes} [Thu Jun 23 16:02:56 2022] POST /identity/v3/projects => generated 385 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 16:02:56.951980 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07431ab-94f4-4d3f-9b5b-630cb63992e5 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:02:56.953790 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-e07431ab-94f4-4d3f-9b5b-630cb63992e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:02:56.961244 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:02:56.972049 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-14d561a1-d33f-4356-93b3-9f5cf474d2d9 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:02:56.973431 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-14d561a1-d33f-4356-93b3-9f5cf474d2d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:02:57.052424 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07431ab-94f4-4d3f-9b5b-630cb63992e5 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:02:57.059265 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ea9a83-1ba7-4b70-9194-65fb60dad33c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:02:57.060200 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ea9a83-1ba7-4b70-9194-65fb60dad33c admin admin] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:57.060415 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ea9a83-1ba7-4b70-9194-65fb60dad33c admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:57.060710 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ea9a83-1ba7-4b70-9194-65fb60dad33c admin admin] PATH_INFO: `/v3/users` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:57.061590 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ea9a83-1ba7-4b70-9194-65fb60dad33c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:02:57.063791 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ea9a83-1ba7-4b70-9194-65fb60dad33c admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:02:57.081589 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1105/2241] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:02:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 16:02:57.096610 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1137/2242] 10.222.0.46 () {60 vars in 1143 bytes} [Thu Jun 23 16:02:56 2022] POST /identity/v3/users => generated 299 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 16:02:57.101041 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d561a1-d33f-4356-93b3-9f5cf474d2d9 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:02:57.114575 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ea9a83-1ba7-4b70-9194-65fb60dad33c admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:02:57.119220 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d561a1-d33f-4356-93b3-9f5cf474d2d9 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:02:57.128357 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-e0ea9a83-1ba7-4b70-9194-65fb60dad33c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:02:57.132112 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058132d1-5e72-4aa6-befd-de0985328d1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:02:57.132838 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058132d1-5e72-4aa6-befd-de0985328d1e admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:57.133140 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058132d1-5e72-4aa6-befd-de0985328d1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:57.133453 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058132d1-5e72-4aa6-befd-de0985328d1e admin admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:57.134231 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058132d1-5e72-4aa6-befd-de0985328d1e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:02:57.135914 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058132d1-5e72-4aa6-befd-de0985328d1e admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:02:57.178217 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1106/2243] 10.222.0.46 () {58 vars in 1121 bytes} [Thu Jun 23 16:02:57 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 16:02:57.182620 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058132d1-5e72-4aa6-befd-de0985328d1e admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:02:57.209409 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058132d1-5e72-4aa6-befd-de0985328d1e admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:02:57.223568 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcbf8e1-2321-4ca0-b7cd-3d4ccc5dbe3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:02:57.224591 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcbf8e1-2321-4ca0-b7cd-3d4ccc5dbe3e admin admin] REQUEST_METHOD: `PUT` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:57.225400 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcbf8e1-2321-4ca0-b7cd-3d4ccc5dbe3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:57.233152 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcbf8e1-2321-4ca0-b7cd-3d4ccc5dbe3e admin admin] PATH_INFO: `/v3/projects/0997f50e0394421f910d58bb2c587d3b/users/c7aff472fdb6435ba267ee3910e517f0/roles/812d3c567c52469cb2b35709fbdc857c` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:57.264037 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fcbf8e1-2321-4ca0-b7cd-3d4ccc5dbe3e admin admin] RBAC: Authorizing `identity:create_grant(project_id=0997f50e0394421f910d58bb2c587d3b, user_id=c7aff472fdb6435ba267ee3910e517f0, role_id=812d3c567c52469cb2b35709fbdc857c)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:02:57.271700 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fcbf8e1-2321-4ca0-b7cd-3d4ccc5dbe3e admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:02:57.277171 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ea9a83-1ba7-4b70-9194-65fb60dad33c admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:02:57.288276 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad86553e-83c2-4e5d-a812-4747b2978aea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51784064951a4619ae09b9167fb0fd0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:02:57.292915 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad86553e-83c2-4e5d-a812-4747b2978aea service neutron] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:57.293243 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad86553e-83c2-4e5d-a812-4747b2978aea service neutron] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:57.293531 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad86553e-83c2-4e5d-a812-4747b2978aea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:57.296417 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1107/2244] 10.222.0.46 () {60 vars in 1482 bytes} [Thu Jun 23 16:02:57 2022] PUT /identity/v3/projects/0997f50e0394421f910d58bb2c587d3b/users/c7aff472fdb6435ba267ee3910e517f0/roles/812d3c567c52469cb2b35709fbdc857c => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 16:02:57.300808 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcbf8e1-2321-4ca0-b7cd-3d4ccc5dbe3e admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:02:57.318296 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad86553e-83c2-4e5d-a812-4747b2978aea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:02:57.319884 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad86553e-83c2-4e5d-a812-4747b2978aea service neutron] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:02:57.343530 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1138/2245] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jun 23 16:02:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 16:02:57.443182 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcbf8e1-2321-4ca0-b7cd-3d4ccc5dbe3e admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:02:57.449949 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d37355-8284-4fb3-b641-d405837c5383 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:02:57.450991 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d37355-8284-4fb3-b641-d405837c5383 admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:57.451455 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d37355-8284-4fb3-b641-d405837c5383 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:57.451887 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d37355-8284-4fb3-b641-d405837c5383 admin admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:57.452976 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d37355-8284-4fb3-b641-d405837c5383 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:02:57.455014 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d37355-8284-4fb3-b641-d405837c5383 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:02:57.472449 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1108/2246] 10.222.0.46 () {58 vars in 1121 bytes} [Thu Jun 23 16:02:57 2022] GET /identity/v3/roles => generated 1420 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 16:02:57.476199 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad86553e-83c2-4e5d-a812-4747b2978aea service neutron] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:02:57.491652 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad86553e-83c2-4e5d-a812-4747b2978aea service neutron] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:02:57.524153 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1f1458-de72-4a26-b213-0af8f4a0b208 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:02:57.525505 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1f1458-de72-4a26-b213-0af8f4a0b208 admin admin] REQUEST_METHOD: `PUT` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:57.525903 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1f1458-de72-4a26-b213-0af8f4a0b208 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:57.526283 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1f1458-de72-4a26-b213-0af8f4a0b208 admin admin] PATH_INFO: `/v3/projects/0997f50e0394421f910d58bb2c587d3b/users/c7aff472fdb6435ba267ee3910e517f0/roles/812d3c567c52469cb2b35709fbdc857c` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:57.528579 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1f1458-de72-4a26-b213-0af8f4a0b208 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0997f50e0394421f910d58bb2c587d3b, user_id=c7aff472fdb6435ba267ee3910e517f0, role_id=812d3c567c52469cb2b35709fbdc857c)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:02:57.531506 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1f1458-de72-4a26-b213-0af8f4a0b208 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:02:57.565680 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1139/2247] 10.222.0.46 () {60 vars in 1482 bytes} [Thu Jun 23 16:02:57 2022] PUT /identity/v3/projects/0997f50e0394421f910d58bb2c587d3b/users/c7aff472fdb6435ba267ee3910e517f0/roles/812d3c567c52469cb2b35709fbdc857c => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 16:02:57.588362 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa74ef0-16a2-4716-86d4-3be4da10e331 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:57.589618 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa74ef0-16a2-4716-86d4-3be4da10e331 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:57.590070 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa74ef0-16a2-4716-86d4-3be4da10e331 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:57.623959 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-faa74ef0-16a2-4716-86d4-3be4da10e331 None None] MFA Rules not processed for user `c7aff472fdb6435ba267ee3910e517f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:02:57.625524 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-faa74ef0-16a2-4716-86d4-3be4da10e331 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:02:57.685408 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:02:57.777569 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1109/2248] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:02:57 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 16:02:57.785705 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1f1458-de72-4a26-b213-0af8f4a0b208 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:02:57.787776 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-6e1f1458-de72-4a26-b213-0af8f4a0b208 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:02:57.906219 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1f1458-de72-4a26-b213-0af8f4a0b208 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:02:57.912087 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d934f1-becc-48af-af54-d2aeeb14d837 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce18eaaba3a6477684fff1fd98ec8b51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:02:57.913050 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d934f1-becc-48af-af54-d2aeeb14d837 service cinder] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:57.913338 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d934f1-becc-48af-af54-d2aeeb14d837 service cinder] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:57.913798 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d934f1-becc-48af-af54-d2aeeb14d837 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:57.916686 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-51d934f1-becc-48af-af54-d2aeeb14d837 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:02:58.014047 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:02:58.074723 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d934f1-becc-48af-af54-d2aeeb14d837 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:02:58.077206 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d934f1-becc-48af-af54-d2aeeb14d837 service cinder] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:02:58.113378 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1140/2249] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jun 23 16:02:57 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 16:02:58.713382 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa74ef0-16a2-4716-86d4-3be4da10e331 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:02:58.714928 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-faa74ef0-16a2-4716-86d4-3be4da10e331 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:02:58.800799 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa74ef0-16a2-4716-86d4-3be4da10e331 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:02:58.810579 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3051d8f8-dc7b-4bac-b9e6-f0292f0aa70b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:02:58.811657 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3051d8f8-dc7b-4bac-b9e6-f0292f0aa70b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:58.812061 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3051d8f8-dc7b-4bac-b9e6-f0292f0aa70b admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:58.812406 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3051d8f8-dc7b-4bac-b9e6-f0292f0aa70b admin admin] PATH_INFO: `/v3/projects/42158893b22d49449502910b10869641` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:58.813738 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3051d8f8-dc7b-4bac-b9e6-f0292f0aa70b admin admin] RBAC: Authorizing `identity:delete_project(project_id=42158893b22d49449502910b10869641)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:02:58.816378 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3051d8f8-dc7b-4bac-b9e6-f0292f0aa70b admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:02:58.878008 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-3051d8f8-dc7b-4bac-b9e6-f0292f0aa70b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '42158893b22d49449502910b10869641', 'request_id': 'req-3051d8f8-dc7b-4bac-b9e6-f0292f0aa70b'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:58.884459 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-3051d8f8-dc7b-4bac-b9e6-f0292f0aa70b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '42158893b22d49449502910b10869641', 'request_id': 'req-3051d8f8-dc7b-4bac-b9e6-f0292f0aa70b'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:58.893052 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-3051d8f8-dc7b-4bac-b9e6-f0292f0aa70b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '42158893b22d49449502910b10869641', 'request_id': 'req-3051d8f8-dc7b-4bac-b9e6-f0292f0aa70b'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:58.911757 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-3051d8f8-dc7b-4bac-b9e6-f0292f0aa70b admin admin] The token cache is being invalidate because project 42158893b22d49449502910b10869641 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80154) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 16:02:58.912317 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-3051d8f8-dc7b-4bac-b9e6-f0292f0aa70b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:58.913290 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-3051d8f8-dc7b-4bac-b9e6-f0292f0aa70b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:02:58.916800 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1110/2250] 10.222.0.46 () {58 vars in 1232 bytes} [Thu Jun 23 16:02:58 2022] DELETE /identity/v3/projects/42158893b22d49449502910b10869641 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 16:02:58.922415 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f567a776-37c0-44fd-b211-8708d7e9dff4 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:58.922914 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f567a776-37c0-44fd-b211-8708d7e9dff4 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:58.923114 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f567a776-37c0-44fd-b211-8708d7e9dff4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:58.965426 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-f567a776-37c0-44fd-b211-8708d7e9dff4 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:02:58.972587 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-f567a776-37c0-44fd-b211-8708d7e9dff4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:02:59.069371 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1141/2251] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:02:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 16:02:59.081298 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef379ff6-03ee-4965-a865-3c053c806d84 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:59.081595 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef379ff6-03ee-4965-a865-3c053c806d84 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:59.081892 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef379ff6-03ee-4965-a865-3c053c806d84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:59.119322 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-ef379ff6-03ee-4965-a865-3c053c806d84 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:02:59.123127 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-ef379ff6-03ee-4965-a865-3c053c806d84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:02:59.152100 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1111/2252] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:02:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 16:02:59.153932 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f567a776-37c0-44fd-b211-8708d7e9dff4 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:02:59.172186 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f567a776-37c0-44fd-b211-8708d7e9dff4 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:02:59.191790 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6b672c-914c-40e2-955a-be1616352b58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:02:59.192560 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6b672c-914c-40e2-955a-be1616352b58 admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:59.192920 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6b672c-914c-40e2-955a-be1616352b58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:59.193266 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6b672c-914c-40e2-955a-be1616352b58 admin admin] PATH_INFO: `/v3/domains` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:59.194656 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6b672c-914c-40e2-955a-be1616352b58 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 16:02:59.194933 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6b672c-914c-40e2-955a-be1616352b58 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:02:59.208326 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6b672c-914c-40e2-955a-be1616352b58 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:02:59.243245 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1142/2253] 10.222.0.46 () {58 vars in 1152 bytes} [Thu Jun 23 16:02:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 16:02:59.249033 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef379ff6-03ee-4965-a865-3c053c806d84 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:02:59.259393 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef379ff6-03ee-4965-a865-3c053c806d84 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:02:59.267094 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e710e96d-2676-4785-9c6a-cbcc2d90c0a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:02:59.268001 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e710e96d-2676-4785-9c6a-cbcc2d90c0a6 admin admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:59.268116 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e710e96d-2676-4785-9c6a-cbcc2d90c0a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:59.268342 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e710e96d-2676-4785-9c6a-cbcc2d90c0a6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:59.269305 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e710e96d-2676-4785-9c6a-cbcc2d90c0a6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:02:59.271120 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e710e96d-2676-4785-9c6a-cbcc2d90c0a6 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:02:59.345110 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1112/2254] 10.222.0.46 () {60 vars in 1152 bytes} [Thu Jun 23 16:02:59 2022] POST /identity/v3/projects => generated 365 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 16:02:59.349244 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6b672c-914c-40e2-955a-be1616352b58 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:02:59.371783 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6b672c-914c-40e2-955a-be1616352b58 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:02:59.453130 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ba5e4c-9bc6-4846-a6af-64c970964912 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:02:59.453842 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ba5e4c-9bc6-4846-a6af-64c970964912 admin admin] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:59.454267 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ba5e4c-9bc6-4846-a6af-64c970964912 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:59.454470 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ba5e4c-9bc6-4846-a6af-64c970964912 admin admin] PATH_INFO: `/v3/users` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:59.455269 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ba5e4c-9bc6-4846-a6af-64c970964912 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:02:59.457646 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ba5e4c-9bc6-4846-a6af-64c970964912 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:02:59.488710 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1143/2255] 10.222.0.46 () {60 vars in 1143 bytes} [Thu Jun 23 16:02:59 2022] POST /identity/v3/users => generated 283 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 16:02:59.489750 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e710e96d-2676-4785-9c6a-cbcc2d90c0a6 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:02:59.503587 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e710e96d-2676-4785-9c6a-cbcc2d90c0a6 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:02:59.510815 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ac8562-d576-4fc1-b1cf-e7d89927bfc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:02:59.511543 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ac8562-d576-4fc1-b1cf-e7d89927bfc1 admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:59.511831 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ac8562-d576-4fc1-b1cf-e7d89927bfc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:59.512112 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ac8562-d576-4fc1-b1cf-e7d89927bfc1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:59.513111 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ac8562-d576-4fc1-b1cf-e7d89927bfc1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:02:59.515357 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ac8562-d576-4fc1-b1cf-e7d89927bfc1 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:02:59.587002 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1113/2256] 10.222.0.46 () {58 vars in 1121 bytes} [Thu Jun 23 16:02:59 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 16:02:59.587108 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ba5e4c-9bc6-4846-a6af-64c970964912 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:02:59.603731 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ba5e4c-9bc6-4846-a6af-64c970964912 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:02:59.611142 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9866ab77-2fe3-4a36-ae21-ea0c5f0e3b0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:02:59.611961 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9866ab77-2fe3-4a36-ae21-ea0c5f0e3b0f admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:59.612334 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9866ab77-2fe3-4a36-ae21-ea0c5f0e3b0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:59.613189 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9866ab77-2fe3-4a36-ae21-ea0c5f0e3b0f admin admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:59.614046 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9866ab77-2fe3-4a36-ae21-ea0c5f0e3b0f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:02:59.616002 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9866ab77-2fe3-4a36-ae21-ea0c5f0e3b0f admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:02:59.647811 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1144/2257] 10.222.0.46 () {58 vars in 1121 bytes} [Thu Jun 23 16:02:59 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 16:02:59.652431 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ac8562-d576-4fc1-b1cf-e7d89927bfc1 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:02:59.684511 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ac8562-d576-4fc1-b1cf-e7d89927bfc1 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:02:59.707122 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4650a6-054f-43f0-b28e-844adfa84bb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:02:59.707122 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4650a6-054f-43f0-b28e-844adfa84bb6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:59.707122 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4650a6-054f-43f0-b28e-844adfa84bb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:59.707122 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4650a6-054f-43f0-b28e-844adfa84bb6 admin admin] PATH_INFO: `/v3/projects/29056405294245a2b4a149bde8bf16da/users/77432115332c4c12a5fe83cb2f9af633/roles/e2a5bbf3200a4d4091d72bc7ff78c34d` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:59.759560 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4650a6-054f-43f0-b28e-844adfa84bb6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=29056405294245a2b4a149bde8bf16da, user_id=77432115332c4c12a5fe83cb2f9af633, role_id=e2a5bbf3200a4d4091d72bc7ff78c34d)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:02:59.769736 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4650a6-054f-43f0-b28e-844adfa84bb6 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:02:59.778312 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:02:59.803536 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1114/2258] 10.222.0.46 () {60 vars in 1482 bytes} [Thu Jun 23 16:02:59 2022] PUT /identity/v3/projects/29056405294245a2b4a149bde8bf16da/users/77432115332c4c12a5fe83cb2f9af633/roles/e2a5bbf3200a4d4091d72bc7ff78c34d => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 16:02:59.898785 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2945ce-e2d1-417d-a756-53659cdf0c7c None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:02:59.899027 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2945ce-e2d1-417d-a756-53659cdf0c7c None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:02:59.899313 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2945ce-e2d1-417d-a756-53659cdf0c7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:02:59.973594 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-0c2945ce-e2d1-417d-a756-53659cdf0c7c None None] MFA Rules not processed for user `77432115332c4c12a5fe83cb2f9af633`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:02:59.975467 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-0c2945ce-e2d1-417d-a756-53659cdf0c7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:03:00.048711 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:03:00.058205 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4650a6-054f-43f0-b28e-844adfa84bb6 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:03:00.063644 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-fe4650a6-054f-43f0-b28e-844adfa84bb6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:03:00.141702 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1145/2259] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:02:59 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 16:03:00.160015 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2945ce-e2d1-417d-a756-53659cdf0c7c None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:03:00.166242 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-0c2945ce-e2d1-417d-a756-53659cdf0c7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:03:00.248045 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4650a6-054f-43f0-b28e-844adfa84bb6 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:03:00.255348 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf97d6f0-21f1-488b-9913-390cb1cf2f95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:03:00.256393 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf97d6f0-21f1-488b-9913-390cb1cf2f95 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:03:00.256690 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf97d6f0-21f1-488b-9913-390cb1cf2f95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:03:00.256998 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf97d6f0-21f1-488b-9913-390cb1cf2f95 admin admin] PATH_INFO: `/v3/users/82e02e90305a424c8e719a92bd084a5f` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:03:00.259113 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf97d6f0-21f1-488b-9913-390cb1cf2f95 admin admin] RBAC: Authorizing `identity:delete_user(user_id=82e02e90305a424c8e719a92bd084a5f)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:03:00.261451 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf97d6f0-21f1-488b-9913-390cb1cf2f95 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:03:00.301618 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2945ce-e2d1-417d-a756-53659cdf0c7c None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:03:00.315270 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed4da84-2663-40e1-9e43-0a032b598fc8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce18eaaba3a6477684fff1fd98ec8b51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:03:00.316036 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed4da84-2663-40e1-9e43-0a032b598fc8 service cinder] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:03:00.316521 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed4da84-2663-40e1-9e43-0a032b598fc8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:03:00.316846 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed4da84-2663-40e1-9e43-0a032b598fc8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:03:00.343349 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-cf97d6f0-21f1-488b-9913-390cb1cf2f95 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '82e02e90305a424c8e719a92bd084a5f'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:03:00.345533 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-cf97d6f0-21f1-488b-9913-390cb1cf2f95 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '82e02e90305a424c8e719a92bd084a5f'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:03:00.355733 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed4da84-2663-40e1-9e43-0a032b598fc8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:03:00.360982 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed4da84-2663-40e1-9e43-0a032b598fc8 service cinder] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:03:00.373027 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-cf97d6f0-21f1-488b-9913-390cb1cf2f95 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '82e02e90305a424c8e719a92bd084a5f'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:03:00.393970 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1146/2260] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jun 23 16:03:00 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 16:03:00.421806 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-cf97d6f0-21f1-488b-9913-390cb1cf2f95 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '82e02e90305a424c8e719a92bd084a5f'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:03:00.424803 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-cf97d6f0-21f1-488b-9913-390cb1cf2f95 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '82e02e90305a424c8e719a92bd084a5f'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:03:00.493720 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1115/2261] 10.222.0.46 () {58 vars in 1223 bytes} [Thu Jun 23 16:03:00 2022] DELETE /identity/v3/users/82e02e90305a424c8e719a92bd084a5f => generated 0 bytes in 436 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 16:03:01.257732 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed4da84-2663-40e1-9e43-0a032b598fc8 service cinder] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:03:01.283362 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed4da84-2663-40e1-9e43-0a032b598fc8 service cinder] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:03:01.418680 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b11b96-47c3-4bfd-91bf-ff1f1e5865d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:03:01.419656 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b11b96-47c3-4bfd-91bf-ff1f1e5865d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:03:01.419941 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b11b96-47c3-4bfd-91bf-ff1f1e5865d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:03:01.420219 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b11b96-47c3-4bfd-91bf-ff1f1e5865d3 admin admin] PATH_INFO: `/v3/projects/27aed572a9da43ca919d77bd579224a9` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:03:01.421803 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b11b96-47c3-4bfd-91bf-ff1f1e5865d3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=27aed572a9da43ca919d77bd579224a9)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:03:01.423820 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b11b96-47c3-4bfd-91bf-ff1f1e5865d3 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:03:01.498623 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-a9b11b96-47c3-4bfd-91bf-ff1f1e5865d3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '27aed572a9da43ca919d77bd579224a9', 'request_id': 'req-a9b11b96-47c3-4bfd-91bf-ff1f1e5865d3'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:03:01.509030 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-a9b11b96-47c3-4bfd-91bf-ff1f1e5865d3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '27aed572a9da43ca919d77bd579224a9', 'request_id': 'req-a9b11b96-47c3-4bfd-91bf-ff1f1e5865d3'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:03:01.519311 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-a9b11b96-47c3-4bfd-91bf-ff1f1e5865d3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '27aed572a9da43ca919d77bd579224a9', 'request_id': 'req-a9b11b96-47c3-4bfd-91bf-ff1f1e5865d3'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:03:01.536962 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-a9b11b96-47c3-4bfd-91bf-ff1f1e5865d3 admin admin] The token cache is being invalidate because project 27aed572a9da43ca919d77bd579224a9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80153) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 16:03:01.537112 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-a9b11b96-47c3-4bfd-91bf-ff1f1e5865d3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:03:01.537996 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-a9b11b96-47c3-4bfd-91bf-ff1f1e5865d3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:03:01.550448 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1147/2262] 10.222.0.46 () {58 vars in 1232 bytes} [Thu Jun 23 16:03:01 2022] DELETE /identity/v3/projects/27aed572a9da43ca919d77bd579224a9 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 16:03:01.558737 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b78820-80be-4d72-bad9-4086e5a02be5 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:03:01.562058 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b78820-80be-4d72-bad9-4086e5a02be5 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:03:01.562359 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b78820-80be-4d72-bad9-4086e5a02be5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:03:01.612141 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-58b78820-80be-4d72-bad9-4086e5a02be5 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:03:01.613707 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-58b78820-80be-4d72-bad9-4086e5a02be5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:03:01.687492 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1116/2263] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:03:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 16:03:01.698878 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9463a823-5cc7-4693-8814-5c663eb309d3 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:03:01.699171 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9463a823-5cc7-4693-8814-5c663eb309d3 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:03:01.699450 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9463a823-5cc7-4693-8814-5c663eb309d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:03:01.723769 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:03:01.741041 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-9463a823-5cc7-4693-8814-5c663eb309d3 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:03:01.743555 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-9463a823-5cc7-4693-8814-5c663eb309d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:03:01.788272 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1148/2264] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:03:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 16:03:01.793254 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b78820-80be-4d72-bad9-4086e5a02be5 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:03:01.814121 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b78820-80be-4d72-bad9-4086e5a02be5 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:03:01.825837 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06dd22b1-624b-48f7-90a6-e8c3987525e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:03:01.826412 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dd22b1-624b-48f7-90a6-e8c3987525e7 admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:03:01.826718 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dd22b1-624b-48f7-90a6-e8c3987525e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:03:01.827317 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dd22b1-624b-48f7-90a6-e8c3987525e7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:03:01.828362 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06dd22b1-624b-48f7-90a6-e8c3987525e7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 16:03:01.828733 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06dd22b1-624b-48f7-90a6-e8c3987525e7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:03:01.836902 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06dd22b1-624b-48f7-90a6-e8c3987525e7 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:03:01.862107 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1117/2265] 10.222.0.46 () {58 vars in 1152 bytes} [Thu Jun 23 16:03:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 16:03:01.866301 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9463a823-5cc7-4693-8814-5c663eb309d3 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:03:01.882416 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9463a823-5cc7-4693-8814-5c663eb309d3 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:03:01.893129 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ae41ec-65b3-4106-864c-60a3ce3dc086 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:03:01.894663 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ae41ec-65b3-4106-864c-60a3ce3dc086 admin admin] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:03:01.895770 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ae41ec-65b3-4106-864c-60a3ce3dc086 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:03:01.896236 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ae41ec-65b3-4106-864c-60a3ce3dc086 admin admin] PATH_INFO: `/v3/projects` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:03:01.897650 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ae41ec-65b3-4106-864c-60a3ce3dc086 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:03:01.902339 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ae41ec-65b3-4106-864c-60a3ce3dc086 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:03:01.959521 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1149/2266] 10.222.0.46 () {60 vars in 1152 bytes} [Thu Jun 23 16:03:01 2022] POST /identity/v3/projects => generated 375 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 16:03:01.959926 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06dd22b1-624b-48f7-90a6-e8c3987525e7 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:03:01.968324 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:03:01.974886 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06dd22b1-624b-48f7-90a6-e8c3987525e7 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:03:02.073138 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd268e09-187b-44e9-b551-01490285c0cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:03:02.074466 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd268e09-187b-44e9-b551-01490285c0cd admin admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:03:02.074466 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd268e09-187b-44e9-b551-01490285c0cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:03:02.075217 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd268e09-187b-44e9-b551-01490285c0cd admin admin] PATH_INFO: `/v3/users` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:03:02.076243 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd268e09-187b-44e9-b551-01490285c0cd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:03:02.078656 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd268e09-187b-44e9-b551-01490285c0cd admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:03:02.109631 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1118/2267] 10.222.0.46 () {60 vars in 1143 bytes} [Thu Jun 23 16:03:01 2022] POST /identity/v3/users => generated 288 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 16:03:02.111435 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ae41ec-65b3-4106-864c-60a3ce3dc086 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:03:02.126427 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ae41ec-65b3-4106-864c-60a3ce3dc086 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:03:02.132583 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3763c046-c7e4-40ca-b4ed-35e6365f67a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:03:02.133436 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3763c046-c7e4-40ca-b4ed-35e6365f67a0 admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:03:02.134221 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3763c046-c7e4-40ca-b4ed-35e6365f67a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:03:02.134221 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3763c046-c7e4-40ca-b4ed-35e6365f67a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:03:02.135503 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3763c046-c7e4-40ca-b4ed-35e6365f67a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:03:02.142746 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3763c046-c7e4-40ca-b4ed-35e6365f67a0 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:03:02.166452 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1150/2268] 10.222.0.46 () {58 vars in 1121 bytes} [Thu Jun 23 16:03:02 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 16:03:02.170729 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd268e09-187b-44e9-b551-01490285c0cd admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:03:02.185773 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd268e09-187b-44e9-b551-01490285c0cd admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:03:02.200765 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00e3c10-7ac5-44e9-8887-d0ba9b9e07b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:03:02.203092 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00e3c10-7ac5-44e9-8887-d0ba9b9e07b5 admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:03:02.203479 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00e3c10-7ac5-44e9-8887-d0ba9b9e07b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:03:02.203827 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00e3c10-7ac5-44e9-8887-d0ba9b9e07b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:03:02.205591 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00e3c10-7ac5-44e9-8887-d0ba9b9e07b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:03:02.209197 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00e3c10-7ac5-44e9-8887-d0ba9b9e07b5 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:03:02.233204 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1119/2269] 10.222.0.46 () {58 vars in 1121 bytes} [Thu Jun 23 16:03:02 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 16:03:02.237448 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3763c046-c7e4-40ca-b4ed-35e6365f67a0 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:03:02.254530 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3763c046-c7e4-40ca-b4ed-35e6365f67a0 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:03:02.266844 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ad81b6-1385-4f81-b065-900225bc91dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:03:02.268081 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ad81b6-1385-4f81-b065-900225bc91dd admin admin] REQUEST_METHOD: `PUT` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:03:02.268580 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ad81b6-1385-4f81-b065-900225bc91dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:03:02.268925 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ad81b6-1385-4f81-b065-900225bc91dd admin admin] PATH_INFO: `/v3/projects/3e09ed68cdf94f5a83ae9581419aa876/users/b423368e84084ac89334f13b723c6b31/roles/e2a5bbf3200a4d4091d72bc7ff78c34d` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:03:02.294230 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ad81b6-1385-4f81-b065-900225bc91dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e09ed68cdf94f5a83ae9581419aa876, user_id=b423368e84084ac89334f13b723c6b31, role_id=e2a5bbf3200a4d4091d72bc7ff78c34d)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:03:02.302543 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ad81b6-1385-4f81-b065-900225bc91dd admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:03:02.319296 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1151/2270] 10.222.0.46 () {60 vars in 1482 bytes} [Thu Jun 23 16:03:02 2022] PUT /identity/v3/projects/3e09ed68cdf94f5a83ae9581419aa876/users/b423368e84084ac89334f13b723c6b31/roles/e2a5bbf3200a4d4091d72bc7ff78c34d => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 16:03:02.325382 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00659c26-ee33-4448-ba59-00767c5c67d1 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:03:02.325681 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00659c26-ee33-4448-ba59-00767c5c67d1 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:03:02.325969 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00659c26-ee33-4448-ba59-00767c5c67d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:03:02.344905 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-00659c26-ee33-4448-ba59-00767c5c67d1 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:03:02.346255 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-00659c26-ee33-4448-ba59-00767c5c67d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:03:02.417133 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1120/2271] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:03:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 16:03:02.427389 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ad81b6-1385-4f81-b065-900225bc91dd admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:03:02.429443 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-c8ad81b6-1385-4f81-b065-900225bc91dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:03:02.514963 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ad81b6-1385-4f81-b065-900225bc91dd admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:03:02.518894 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1ea97b-fa08-4ac2-a199-11f0e493b4ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51784064951a4619ae09b9167fb0fd0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:03:02.519556 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1ea97b-fa08-4ac2-a199-11f0e493b4ef service neutron] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:03:02.519859 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1ea97b-fa08-4ac2-a199-11f0e493b4ef service neutron] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:03:02.520125 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1ea97b-fa08-4ac2-a199-11f0e493b4ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:03:02.533035 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1ea97b-fa08-4ac2-a199-11f0e493b4ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:03:02.534708 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1ea97b-fa08-4ac2-a199-11f0e493b4ef service neutron] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:03:02.546766 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1152/2272] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jun 23 16:03:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 16:03:04.521644 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f995db-527b-4596-9864-c82ff52d2866 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:03:04.522927 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f995db-527b-4596-9864-c82ff52d2866 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:03:04.523260 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f995db-527b-4596-9864-c82ff52d2866 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:03:04.527831 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1ea97b-fa08-4ac2-a199-11f0e493b4ef service neutron] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:03:04.529175 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-2e1ea97b-fa08-4ac2-a199-11f0e493b4ef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:03:04.608889 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-30f995db-527b-4596-9864-c82ff52d2866 None None] MFA Rules not processed for user `2399b4fc64f74475bd8dc7b1634a2c2a`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:03:04.610135 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-30f995db-527b-4596-9864-c82ff52d2866 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:03:04.680538 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1ea97b-fa08-4ac2-a199-11f0e493b4ef service neutron] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:03:04.689851 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfdc6e8-7cf1-4c4c-8d17-d3f9c8d4f707 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce18eaaba3a6477684fff1fd98ec8b51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:03:04.690989 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfdc6e8-7cf1-4c4c-8d17-d3f9c8d4f707 service cinder] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:03:04.691479 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfdc6e8-7cf1-4c4c-8d17-d3f9c8d4f707 service cinder] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:03:04.691951 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfdc6e8-7cf1-4c4c-8d17-d3f9c8d4f707 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:03:04.695127 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-9dfdc6e8-7cf1-4c4c-8d17-d3f9c8d4f707 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:03:04.732982 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1121/2273] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:03:04 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 16:03:04.734614 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f995db-527b-4596-9864-c82ff52d2866 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:03:04.737155 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-30f995db-527b-4596-9864-c82ff52d2866 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:03:04.758299 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f995db-527b-4596-9864-c82ff52d2866 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:03:04.768941 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9d274c-b926-41b7-a5f9-94ab558663b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:03:04.770475 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9d274c-b926-41b7-a5f9-94ab558663b9 admin admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:03:04.771420 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9d274c-b926-41b7-a5f9-94ab558663b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:03:04.771946 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9d274c-b926-41b7-a5f9-94ab558663b9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:03:04.772980 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9d274c-b926-41b7-a5f9-94ab558663b9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:03:04.779739 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9d274c-b926-41b7-a5f9-94ab558663b9 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:03:04.815476 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:03:04.873198 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1122/2274] 10.222.0.46 () {60 vars in 1152 bytes} [Thu Jun 23 16:03:04 2022] POST /identity/v3/projects => generated 371 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 16:03:04.876153 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:03:04.877098 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9d274c-b926-41b7-a5f9-94ab558663b9 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:03:04.883346 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfdc6e8-7cf1-4c4c-8d17-d3f9c8d4f707 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:03:04.885694 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfdc6e8-7cf1-4c4c-8d17-d3f9c8d4f707 service cinder] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:03:04.910557 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1153/2275] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jun 23 16:03:04 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 16:03:05.069657 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9d274c-b926-41b7-a5f9-94ab558663b9 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:03:05.082153 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07397cad-5dc3-4fc2-8f4f-b4349bfdad44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:03:05.083759 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07397cad-5dc3-4fc2-8f4f-b4349bfdad44 admin admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:03:05.084446 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07397cad-5dc3-4fc2-8f4f-b4349bfdad44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:03:05.088027 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07397cad-5dc3-4fc2-8f4f-b4349bfdad44 admin admin] PATH_INFO: `/v3/users` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:03:05.089703 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07397cad-5dc3-4fc2-8f4f-b4349bfdad44 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:03:05.092120 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07397cad-5dc3-4fc2-8f4f-b4349bfdad44 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:03:05.150676 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1123/2276] 10.222.0.46 () {60 vars in 1143 bytes} [Thu Jun 23 16:03:04 2022] POST /identity/v3/users => generated 292 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 16:03:05.155622 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfdc6e8-7cf1-4c4c-8d17-d3f9c8d4f707 service cinder] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:03:05.207001 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfdc6e8-7cf1-4c4c-8d17-d3f9c8d4f707 service cinder] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:03:05.224304 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad1a02a-1d6e-4df8-ba72-e26d26080cd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:03:05.226624 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad1a02a-1d6e-4df8-ba72-e26d26080cd3 admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:03:05.228603 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad1a02a-1d6e-4df8-ba72-e26d26080cd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:03:05.230304 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad1a02a-1d6e-4df8-ba72-e26d26080cd3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:03:05.232733 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad1a02a-1d6e-4df8-ba72-e26d26080cd3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:03:05.240248 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad1a02a-1d6e-4df8-ba72-e26d26080cd3 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:03:05.276266 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1154/2277] 10.222.0.46 () {58 vars in 1121 bytes} [Thu Jun 23 16:03:05 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 16:03:05.280615 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07397cad-5dc3-4fc2-8f4f-b4349bfdad44 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:03:05.299239 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07397cad-5dc3-4fc2-8f4f-b4349bfdad44 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:03:05.320878 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ca2dea-c578-4cd6-be0b-fbfaac9530a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:03:05.321891 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ca2dea-c578-4cd6-be0b-fbfaac9530a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:03:05.322219 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ca2dea-c578-4cd6-be0b-fbfaac9530a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:03:05.322463 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ca2dea-c578-4cd6-be0b-fbfaac9530a5 admin admin] PATH_INFO: `/v3/projects/5646db2a9be34515a21840c4569b795b/users/8969b7af3a6d48019b419a5eb4a8846d/roles/812d3c567c52469cb2b35709fbdc857c` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:03:05.356168 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ca2dea-c578-4cd6-be0b-fbfaac9530a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5646db2a9be34515a21840c4569b795b, user_id=8969b7af3a6d48019b419a5eb4a8846d, role_id=812d3c567c52469cb2b35709fbdc857c)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:03:05.362052 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ca2dea-c578-4cd6-be0b-fbfaac9530a5 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:03:05.410225 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1124/2278] 10.222.0.46 () {60 vars in 1482 bytes} [Thu Jun 23 16:03:05 2022] PUT /identity/v3/projects/5646db2a9be34515a21840c4569b795b/users/8969b7af3a6d48019b419a5eb4a8846d/roles/812d3c567c52469cb2b35709fbdc857c => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 16:03:05.415528 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad1a02a-1d6e-4df8-ba72-e26d26080cd3 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:03:05.442981 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad1a02a-1d6e-4df8-ba72-e26d26080cd3 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:03:05.636060 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c86dc5d-5813-43ad-b333-21948e8ed1e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:03:05.636060 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c86dc5d-5813-43ad-b333-21948e8ed1e0 admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:03:05.636060 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c86dc5d-5813-43ad-b333-21948e8ed1e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:03:05.636060 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c86dc5d-5813-43ad-b333-21948e8ed1e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:03:05.638625 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c86dc5d-5813-43ad-b333-21948e8ed1e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:03:05.638625 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c86dc5d-5813-43ad-b333-21948e8ed1e0 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:03:05.678877 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ca2dea-c578-4cd6-be0b-fbfaac9530a5 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:03:05.690006 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1155/2279] 10.222.0.46 () {58 vars in 1121 bytes} [Thu Jun 23 16:03:05 2022] GET /identity/v3/roles => generated 1420 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 16:03:05.703959 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ca2dea-c578-4cd6-be0b-fbfaac9530a5 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:03:05.727915 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f4a7b7-77bd-402b-811a-9265fb4c48f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:03:05.729573 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f4a7b7-77bd-402b-811a-9265fb4c48f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:03:05.730107 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f4a7b7-77bd-402b-811a-9265fb4c48f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:03:05.730591 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f4a7b7-77bd-402b-811a-9265fb4c48f3 admin admin] PATH_INFO: `/v3/projects/5646db2a9be34515a21840c4569b795b/users/8969b7af3a6d48019b419a5eb4a8846d/roles/812d3c567c52469cb2b35709fbdc857c` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:03:05.751260 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f4a7b7-77bd-402b-811a-9265fb4c48f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5646db2a9be34515a21840c4569b795b, user_id=8969b7af3a6d48019b419a5eb4a8846d, role_id=812d3c567c52469cb2b35709fbdc857c)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:03:05.751260 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f4a7b7-77bd-402b-811a-9265fb4c48f3 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:03:05.761757 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1125/2280] 10.222.0.46 () {60 vars in 1482 bytes} [Thu Jun 23 16:03:05 2022] PUT /identity/v3/projects/5646db2a9be34515a21840c4569b795b/users/8969b7af3a6d48019b419a5eb4a8846d/roles/812d3c567c52469cb2b35709fbdc857c => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 16:03:10.987984 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c86dc5d-5813-43ad-b333-21948e8ed1e0 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:03:10.990221 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-9c86dc5d-5813-43ad-b333-21948e8ed1e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:03:11.024121 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46de57f-01da-42e9-8714-1a4af340531d None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:03:11.024584 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46de57f-01da-42e9-8714-1a4af340531d None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:03:11.025316 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46de57f-01da-42e9-8714-1a4af340531d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:03:11.088250 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-e46de57f-01da-42e9-8714-1a4af340531d None None] MFA Rules not processed for user `ad8dac501e254a8d9671083b7b7d413c`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:03:11.090042 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-e46de57f-01da-42e9-8714-1a4af340531d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:03:11.299782 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c86dc5d-5813-43ad-b333-21948e8ed1e0 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:03:11.306042 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1126/2281] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:03:11 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 284 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 16:03:11.520073 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91913300-b1e5-42fc-b1f6-d5111c09b485 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:03:11.521262 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91913300-b1e5-42fc-b1f6-d5111c09b485 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:03:11.521706 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91913300-b1e5-42fc-b1f6-d5111c09b485 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:03:11.522204 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91913300-b1e5-42fc-b1f6-d5111c09b485 admin admin] PATH_INFO: `/v3/users/c5626c4a29674aaea2469595002260ae` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:03:11.526063 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91913300-b1e5-42fc-b1f6-d5111c09b485 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c5626c4a29674aaea2469595002260ae)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:03:11.528446 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91913300-b1e5-42fc-b1f6-d5111c09b485 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:03:11.597154 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:03:11.657618 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-91913300-b1e5-42fc-b1f6-d5111c09b485 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c5626c4a29674aaea2469595002260ae'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:03:11.680878 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-91913300-b1e5-42fc-b1f6-d5111c09b485 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c5626c4a29674aaea2469595002260ae'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:03:11.681363 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-91913300-b1e5-42fc-b1f6-d5111c09b485 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c5626c4a29674aaea2469595002260ae'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:03:11.683010 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-91913300-b1e5-42fc-b1f6-d5111c09b485 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c5626c4a29674aaea2469595002260ae'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:03:11.722211 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-91913300-b1e5-42fc-b1f6-d5111c09b485 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c5626c4a29674aaea2469595002260ae'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:03:11.745624 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1156/2282] 10.222.0.46 () {58 vars in 1223 bytes} [Thu Jun 23 16:03:10 2022] DELETE /identity/v3/users/c5626c4a29674aaea2469595002260ae => generated 0 bytes in 764 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 16:03:11.749969 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de33b77d-6db6-4774-b18f-9e055b972670 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:03:11.750264 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de33b77d-6db6-4774-b18f-9e055b972670 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:03:11.750546 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de33b77d-6db6-4774-b18f-9e055b972670 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:03:11.808671 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-de33b77d-6db6-4774-b18f-9e055b972670 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:03:11.809942 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-de33b77d-6db6-4774-b18f-9e055b972670 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:03:11.870009 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:03:12.003808 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1127/2283] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:03:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 16:03:12.131250 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91913300-b1e5-42fc-b1f6-d5111c09b485 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:03:12.134615 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-91913300-b1e5-42fc-b1f6-d5111c09b485 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:03:12.407667 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91913300-b1e5-42fc-b1f6-d5111c09b485 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:03:12.419080 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fe5d19-6683-4dbb-8e9f-8a8d05a57baf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51784064951a4619ae09b9167fb0fd0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:03:12.419975 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fe5d19-6683-4dbb-8e9f-8a8d05a57baf service neutron] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:03:12.420424 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fe5d19-6683-4dbb-8e9f-8a8d05a57baf service neutron] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:03:12.420846 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fe5d19-6683-4dbb-8e9f-8a8d05a57baf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:03:12.457557 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fe5d19-6683-4dbb-8e9f-8a8d05a57baf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:03:12.459844 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fe5d19-6683-4dbb-8e9f-8a8d05a57baf service neutron] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:03:12.487415 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1157/2284] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jun 23 16:03:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 16:03:13.934493 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de33b77d-6db6-4774-b18f-9e055b972670 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:03:13.941527 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-de33b77d-6db6-4774-b18f-9e055b972670 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:03:13.989483 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de33b77d-6db6-4774-b18f-9e055b972670 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:03:14.013864 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754a01b2-3f69-4404-8f39-85ae29288537 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:03:14.016228 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754a01b2-3f69-4404-8f39-85ae29288537 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:03:14.018630 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754a01b2-3f69-4404-8f39-85ae29288537 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:03:14.019791 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754a01b2-3f69-4404-8f39-85ae29288537 admin admin] PATH_INFO: `/v3/projects/3c0491559ff54189be425d60c5c4f36c` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:03:14.022837 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754a01b2-3f69-4404-8f39-85ae29288537 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3c0491559ff54189be425d60c5c4f36c)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:03:14.026974 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754a01b2-3f69-4404-8f39-85ae29288537 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:03:14.130629 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-754a01b2-3f69-4404-8f39-85ae29288537 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3c0491559ff54189be425d60c5c4f36c', 'request_id': 'req-754a01b2-3f69-4404-8f39-85ae29288537'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:03:14.138110 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-754a01b2-3f69-4404-8f39-85ae29288537 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3c0491559ff54189be425d60c5c4f36c', 'request_id': 'req-754a01b2-3f69-4404-8f39-85ae29288537'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:03:14.154387 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-754a01b2-3f69-4404-8f39-85ae29288537 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3c0491559ff54189be425d60c5c4f36c', 'request_id': 'req-754a01b2-3f69-4404-8f39-85ae29288537'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:03:14.169475 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc61bb3-6da7-47d4-84ba-ed45e79ddfc4 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:03:14.170388 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc61bb3-6da7-47d4-84ba-ed45e79ddfc4 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:03:14.170897 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc61bb3-6da7-47d4-84ba-ed45e79ddfc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:03:14.203701 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-754a01b2-3f69-4404-8f39-85ae29288537 admin admin] The token cache is being invalidate because project 3c0491559ff54189be425d60c5c4f36c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80154) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 16:03:14.204016 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-754a01b2-3f69-4404-8f39-85ae29288537 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:03:14.204851 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-754a01b2-3f69-4404-8f39-85ae29288537 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:03:14.207690 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1128/2285] 10.222.0.46 () {58 vars in 1232 bytes} [Thu Jun 23 16:03:13 2022] DELETE /identity/v3/projects/3c0491559ff54189be425d60c5c4f36c => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 16:03:14.216017 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754a01b2-3f69-4404-8f39-85ae29288537 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:03:14.221647 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-754a01b2-3f69-4404-8f39-85ae29288537 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:03:14.234209 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-9bc61bb3-6da7-47d4-84ba-ed45e79ddfc4 None None] MFA Rules not processed for user `8969b7af3a6d48019b419a5eb4a8846d`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:03:14.235595 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-9bc61bb3-6da7-47d4-84ba-ed45e79ddfc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:03:14.348306 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754a01b2-3f69-4404-8f39-85ae29288537 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:03:14.366181 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:03:14.368460 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaabe2bd-3a47-410a-8771-e484e2a89e1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:03:14.377856 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaabe2bd-3a47-410a-8771-e484e2a89e1e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:03:14.378355 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaabe2bd-3a47-410a-8771-e484e2a89e1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:03:14.378788 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaabe2bd-3a47-410a-8771-e484e2a89e1e admin admin] PATH_INFO: `/v3/users/c7aff472fdb6435ba267ee3910e517f0` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:03:14.383449 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaabe2bd-3a47-410a-8771-e484e2a89e1e admin admin] RBAC: Authorizing `identity:delete_user(user_id=c7aff472fdb6435ba267ee3910e517f0)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:03:14.389662 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaabe2bd-3a47-410a-8771-e484e2a89e1e admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:03:14.409172 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1158/2286] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:03:14 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 16:03:14.411477 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6555d265-bb73-4d5f-941e-42a8da424c4d None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:03:14.411936 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6555d265-bb73-4d5f-941e-42a8da424c4d None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:03:14.412346 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6555d265-bb73-4d5f-941e-42a8da424c4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:03:14.457980 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-6555d265-bb73-4d5f-941e-42a8da424c4d None None] MFA Rules not processed for user `b423368e84084ac89334f13b723c6b31`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:03:14.461600 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-6555d265-bb73-4d5f-941e-42a8da424c4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:03:14.511378 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-eaabe2bd-3a47-410a-8771-e484e2a89e1e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c7aff472fdb6435ba267ee3910e517f0'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:03:14.512509 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-eaabe2bd-3a47-410a-8771-e484e2a89e1e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c7aff472fdb6435ba267ee3910e517f0'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:03:14.520384 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:03:14.523572 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-eaabe2bd-3a47-410a-8771-e484e2a89e1e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c7aff472fdb6435ba267ee3910e517f0'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:03:14.536328 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-eaabe2bd-3a47-410a-8771-e484e2a89e1e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c7aff472fdb6435ba267ee3910e517f0'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:03:14.539258 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-eaabe2bd-3a47-410a-8771-e484e2a89e1e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c7aff472fdb6435ba267ee3910e517f0'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:03:14.607654 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1159/2287] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:03:14 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 16:03:14.608791 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6555d265-bb73-4d5f-941e-42a8da424c4d None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:03:14.609167 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1129/2288] 10.222.0.46 () {58 vars in 1223 bytes} [Thu Jun 23 16:03:14 2022] DELETE /identity/v3/users/c7aff472fdb6435ba267ee3910e517f0 => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 16:03:14.613216 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-6555d265-bb73-4d5f-941e-42a8da424c4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:03:14.655160 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaabe2bd-3a47-410a-8771-e484e2a89e1e admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:03:14.657309 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-eaabe2bd-3a47-410a-8771-e484e2a89e1e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:03:14.766773 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaabe2bd-3a47-410a-8771-e484e2a89e1e admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:03:14.769678 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6555d265-bb73-4d5f-941e-42a8da424c4d None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:03:14.777965 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a642524d-2463-46fe-b531-02f7411d38bd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '705115d19bab424d894bea8b1b48a7f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:03:14.779851 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613b4b1b-c714-4df5-92cd-eb243eddc048 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '705115d19bab424d894bea8b1b48a7f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:03:14.780888 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a642524d-2463-46fe-b531-02f7411d38bd service nova] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:03:14.781494 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a642524d-2463-46fe-b531-02f7411d38bd service nova] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:03:14.781922 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613b4b1b-c714-4df5-92cd-eb243eddc048 service nova] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:03:14.782275 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a642524d-2463-46fe-b531-02f7411d38bd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:03:14.782642 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613b4b1b-c714-4df5-92cd-eb243eddc048 service nova] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:03:14.783203 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613b4b1b-c714-4df5-92cd-eb243eddc048 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:03:14.786089 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-613b4b1b-c714-4df5-92cd-eb243eddc048 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:03:14.885277 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a642524d-2463-46fe-b531-02f7411d38bd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:03:14.887448 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a642524d-2463-46fe-b531-02f7411d38bd service nova] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:03:14.901774 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613b4b1b-c714-4df5-92cd-eb243eddc048 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:03:14.904438 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1130/2289] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jun 23 16:03:14 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 250 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 16:03:14.904569 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613b4b1b-c714-4df5-92cd-eb243eddc048 service nova] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:03:14.916392 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1160/2290] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jun 23 16:03:14 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 16:03:15.160137 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a642524d-2463-46fe-b531-02f7411d38bd service nova] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:03:15.161695 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-a642524d-2463-46fe-b531-02f7411d38bd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:03:15.308337 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a642524d-2463-46fe-b531-02f7411d38bd service nova] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:03:15.316032 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28dbb56-4aa3-4a81-9a8c-02bf8e7f0160 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:03:15.317137 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28dbb56-4aa3-4a81-9a8c-02bf8e7f0160 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:03:15.317570 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28dbb56-4aa3-4a81-9a8c-02bf8e7f0160 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:03:15.318180 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28dbb56-4aa3-4a81-9a8c-02bf8e7f0160 admin admin] PATH_INFO: `/v3/projects/0997f50e0394421f910d58bb2c587d3b` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:03:15.320294 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28dbb56-4aa3-4a81-9a8c-02bf8e7f0160 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0997f50e0394421f910d58bb2c587d3b)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:03:15.323673 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28dbb56-4aa3-4a81-9a8c-02bf8e7f0160 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:03:15.333002 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:03:15.463169 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-e28dbb56-4aa3-4a81-9a8c-02bf8e7f0160 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0997f50e0394421f910d58bb2c587d3b', 'request_id': 'req-e28dbb56-4aa3-4a81-9a8c-02bf8e7f0160'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:03:15.475210 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-e28dbb56-4aa3-4a81-9a8c-02bf8e7f0160 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0997f50e0394421f910d58bb2c587d3b', 'request_id': 'req-e28dbb56-4aa3-4a81-9a8c-02bf8e7f0160'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:03:15.516748 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-e28dbb56-4aa3-4a81-9a8c-02bf8e7f0160 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0997f50e0394421f910d58bb2c587d3b', 'request_id': 'req-e28dbb56-4aa3-4a81-9a8c-02bf8e7f0160'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:03:15.541673 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-e28dbb56-4aa3-4a81-9a8c-02bf8e7f0160 admin admin] The token cache is being invalidate because project 0997f50e0394421f910d58bb2c587d3b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80154) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 16:03:15.542183 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-e28dbb56-4aa3-4a81-9a8c-02bf8e7f0160 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:03:15.545105 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-e28dbb56-4aa3-4a81-9a8c-02bf8e7f0160 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:03:15.547492 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1131/2291] 10.222.0.46 () {58 vars in 1232 bytes} [Thu Jun 23 16:03:15 2022] DELETE /identity/v3/projects/0997f50e0394421f910d58bb2c587d3b => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 16:03:15.564008 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022d747d-00d1-426b-a085-d128fc645d3b None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:03:15.565297 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022d747d-00d1-426b-a085-d128fc645d3b None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:03:15.568070 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022d747d-00d1-426b-a085-d128fc645d3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:03:15.642090 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:03:15.663041 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-022d747d-00d1-426b-a085-d128fc645d3b None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:03:15.664936 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-022d747d-00d1-426b-a085-d128fc645d3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:03:15.764591 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1161/2292] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:03:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 16:03:15.775384 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ab0b67-23e8-438b-87e5-c31d06c3f3aa None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:03:15.775679 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ab0b67-23e8-438b-87e5-c31d06c3f3aa None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:03:15.775953 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ab0b67-23e8-438b-87e5-c31d06c3f3aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:03:15.807859 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-77ab0b67-23e8-438b-87e5-c31d06c3f3aa None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:03:15.809266 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-77ab0b67-23e8-438b-87e5-c31d06c3f3aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:03:15.822108 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1132/2293] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:03:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 16:03:15.823452 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022d747d-00d1-426b-a085-d128fc645d3b None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:03:15.834130 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022d747d-00d1-426b-a085-d128fc645d3b None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:03:15.846083 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdf45a0-9513-41ae-ad7b-c24375f08b32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:03:15.846852 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdf45a0-9513-41ae-ad7b-c24375f08b32 admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:03:15.847091 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdf45a0-9513-41ae-ad7b-c24375f08b32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:03:15.847371 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdf45a0-9513-41ae-ad7b-c24375f08b32 admin admin] PATH_INFO: `/v3/domains` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:03:15.848124 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdf45a0-9513-41ae-ad7b-c24375f08b32 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 16:03:15.848432 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdf45a0-9513-41ae-ad7b-c24375f08b32 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:03:15.850088 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdf45a0-9513-41ae-ad7b-c24375f08b32 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:03:15.863985 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1162/2294] 10.222.0.46 () {58 vars in 1152 bytes} [Thu Jun 23 16:03:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 16:03:15.868944 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ab0b67-23e8-438b-87e5-c31d06c3f3aa None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:03:15.881981 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ab0b67-23e8-438b-87e5-c31d06c3f3aa None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:03:15.900682 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79f0ae7-c70c-4748-85e0-a68ede293b32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:03:15.901371 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79f0ae7-c70c-4748-85e0-a68ede293b32 admin admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:03:15.901655 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79f0ae7-c70c-4748-85e0-a68ede293b32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:03:15.901930 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79f0ae7-c70c-4748-85e0-a68ede293b32 admin admin] PATH_INFO: `/v3/projects` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:03:15.902938 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79f0ae7-c70c-4748-85e0-a68ede293b32 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:03:15.904666 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79f0ae7-c70c-4748-85e0-a68ede293b32 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:03:15.935137 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1133/2295] 10.222.0.46 () {60 vars in 1152 bytes} [Thu Jun 23 16:03:15 2022] POST /identity/v3/projects => generated 371 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 16:03:15.941380 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdf45a0-9513-41ae-ad7b-c24375f08b32 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:03:15.954847 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdf45a0-9513-41ae-ad7b-c24375f08b32 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:03:16.039191 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb0a9d8-dbaa-4d74-8d54-d0b8ae81022a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:03:16.039918 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb0a9d8-dbaa-4d74-8d54-d0b8ae81022a admin admin] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:03:16.040203 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb0a9d8-dbaa-4d74-8d54-d0b8ae81022a admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:03:16.040673 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb0a9d8-dbaa-4d74-8d54-d0b8ae81022a admin admin] PATH_INFO: `/v3/users` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:03:16.041757 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb0a9d8-dbaa-4d74-8d54-d0b8ae81022a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:03:16.043779 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb0a9d8-dbaa-4d74-8d54-d0b8ae81022a admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:03:16.074747 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1163/2296] 10.222.0.46 () {60 vars in 1143 bytes} [Thu Jun 23 16:03:15 2022] POST /identity/v3/users => generated 286 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 16:03:16.078293 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79f0ae7-c70c-4748-85e0-a68ede293b32 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:03:16.092361 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79f0ae7-c70c-4748-85e0-a68ede293b32 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:03:16.099528 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71dd6c12-6a31-4b07-b1f9-c93e4f77c757 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:03:16.100224 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dd6c12-6a31-4b07-b1f9-c93e4f77c757 admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:03:16.100531 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dd6c12-6a31-4b07-b1f9-c93e4f77c757 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:03:16.100786 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dd6c12-6a31-4b07-b1f9-c93e4f77c757 admin admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:03:16.101726 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71dd6c12-6a31-4b07-b1f9-c93e4f77c757 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:03:16.103530 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71dd6c12-6a31-4b07-b1f9-c93e4f77c757 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:03:16.117409 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1134/2297] 10.222.0.46 () {58 vars in 1121 bytes} [Thu Jun 23 16:03:16 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 16:03:16.121602 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb0a9d8-dbaa-4d74-8d54-d0b8ae81022a admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:03:16.133602 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb0a9d8-dbaa-4d74-8d54-d0b8ae81022a admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:03:16.138435 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eeab8c7-b132-49f6-9252-3eb3fab1626d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:03:16.139184 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eeab8c7-b132-49f6-9252-3eb3fab1626d admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:03:16.139477 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eeab8c7-b132-49f6-9252-3eb3fab1626d admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:03:16.139760 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eeab8c7-b132-49f6-9252-3eb3fab1626d admin admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:03:16.140530 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eeab8c7-b132-49f6-9252-3eb3fab1626d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:03:16.142196 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eeab8c7-b132-49f6-9252-3eb3fab1626d admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:03:16.157444 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1164/2298] 10.222.0.46 () {58 vars in 1121 bytes} [Thu Jun 23 16:03:16 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 16:03:16.161480 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71dd6c12-6a31-4b07-b1f9-c93e4f77c757 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:03:16.178912 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71dd6c12-6a31-4b07-b1f9-c93e4f77c757 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:03:16.186400 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3e6fce-420c-4d58-94a6-7a804f90c667 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:03:16.187388 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3e6fce-420c-4d58-94a6-7a804f90c667 admin admin] REQUEST_METHOD: `PUT` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:03:16.188197 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3e6fce-420c-4d58-94a6-7a804f90c667 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:03:16.188197 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3e6fce-420c-4d58-94a6-7a804f90c667 admin admin] PATH_INFO: `/v3/projects/9710e33d284d4851b5b7cc47b09546df/users/e805623e339a46db99b7040fd2a6069e/roles/e2a5bbf3200a4d4091d72bc7ff78c34d` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:03:16.208277 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3e6fce-420c-4d58-94a6-7a804f90c667 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9710e33d284d4851b5b7cc47b09546df, user_id=e805623e339a46db99b7040fd2a6069e, role_id=e2a5bbf3200a4d4091d72bc7ff78c34d)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:03:16.211194 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3e6fce-420c-4d58-94a6-7a804f90c667 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:03:16.235533 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1135/2299] 10.222.0.46 () {60 vars in 1482 bytes} [Thu Jun 23 16:03:16 2022] PUT /identity/v3/projects/9710e33d284d4851b5b7cc47b09546df/users/e805623e339a46db99b7040fd2a6069e/roles/e2a5bbf3200a4d4091d72bc7ff78c34d => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 16:03:16.247777 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30086c20-8404-4361-9be4-c620d511ff1b None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:03:16.248077 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30086c20-8404-4361-9be4-c620d511ff1b None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:03:16.248360 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30086c20-8404-4361-9be4-c620d511ff1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:03:16.286048 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-30086c20-8404-4361-9be4-c620d511ff1b None None] MFA Rules not processed for user `e805623e339a46db99b7040fd2a6069e`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:03:16.288247 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-30086c20-8404-4361-9be4-c620d511ff1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:03:16.386387 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:03:16.394580 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1165/2300] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:03:16 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 16:03:16.410913 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3e6fce-420c-4d58-94a6-7a804f90c667 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:03:16.413585 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-cd3e6fce-420c-4d58-94a6-7a804f90c667 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:03:16.512157 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3e6fce-420c-4d58-94a6-7a804f90c667 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:03:16.524637 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd49ef1a-5453-44f8-b918-9675152b5eab service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83970271286e451e9008bbf07cd3fa86', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:03:16.527763 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd49ef1a-5453-44f8-b918-9675152b5eab service glance] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:03:16.528066 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd49ef1a-5453-44f8-b918-9675152b5eab service glance] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:03:16.528350 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd49ef1a-5453-44f8-b918-9675152b5eab service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:03:16.549161 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd49ef1a-5453-44f8-b918-9675152b5eab service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:03:16.550617 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd49ef1a-5453-44f8-b918-9675152b5eab service glance] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:03:16.565554 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1136/2301] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jun 23 16:03:16 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 16:03:16.672327 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c486af-87b9-40d2-b58f-8c8061e00b95 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:03:16.672801 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c486af-87b9-40d2-b58f-8c8061e00b95 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:03:16.672895 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c486af-87b9-40d2-b58f-8c8061e00b95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:03:16.703046 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-c0c486af-87b9-40d2-b58f-8c8061e00b95 None None] MFA Rules not processed for user `2399b4fc64f74475bd8dc7b1634a2c2a`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:03:16.704284 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-c0c486af-87b9-40d2-b58f-8c8061e00b95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:03:16.800527 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1166/2302] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:03:16 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 16:03:17.237226 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf605773-d759-4a4f-90a3-93f6f91db5c3 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:03:17.237704 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf605773-d759-4a4f-90a3-93f6f91db5c3 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:03:17.237813 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf605773-d759-4a4f-90a3-93f6f91db5c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:03:17.261039 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-cf605773-d759-4a4f-90a3-93f6f91db5c3 None None] MFA Rules not processed for user `b423368e84084ac89334f13b723c6b31`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:03:17.262798 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-cf605773-d759-4a4f-90a3-93f6f91db5c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:03:17.318033 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:03:17.373804 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1137/2303] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:03:17 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 16:03:26.791091 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=26 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:03:27.363437 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=9 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:03:35.874833 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34ae80e-98ac-4f07-8cb6-69d15a671f42 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:03:35.875449 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34ae80e-98ac-4f07-8cb6-69d15a671f42 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:03:35.875449 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34ae80e-98ac-4f07-8cb6-69d15a671f42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:03:35.893581 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-e34ae80e-98ac-4f07-8cb6-69d15a671f42 None None] MFA Rules not processed for user `228e5acf4578494ab1a28e4f530233f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:03:35.895248 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-e34ae80e-98ac-4f07-8cb6-69d15a671f42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:03:35.951927 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1167/2304] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:03:35 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 16:03:45.944692 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=25 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:04:01.065389 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73df4230-bd03-4b7a-bb42-6822dc52b212 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:04:01.065970 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73df4230-bd03-4b7a-bb42-6822dc52b212 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:04:01.065970 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73df4230-bd03-4b7a-bb42-6822dc52b212 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:04:01.104901 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-73df4230-bd03-4b7a-bb42-6822dc52b212 None None] MFA Rules not processed for user `be48addfaa154555b8d0cb7aac8b32ac`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:04:01.106310 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-73df4230-bd03-4b7a-bb42-6822dc52b212 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:04:01.191038 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1138/2305] 10.222.0.46 () {58 vars in 1009 bytes} [Thu Jun 23 16:04:01 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 16:04:01.244625 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34ae80e-98ac-4f07-8cb6-69d15a671f42 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:04:01.246080 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-e34ae80e-98ac-4f07-8cb6-69d15a671f42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:04:01.321557 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34ae80e-98ac-4f07-8cb6-69d15a671f42 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:04:01.326821 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729c9e2d-f013-4c7e-9001-ed2244ef25d1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '242bf02d14724fe4837fcce8730377f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:04:01.327644 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729c9e2d-f013-4c7e-9001-ed2244ef25d1 service swift] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:04:01.327931 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729c9e2d-f013-4c7e-9001-ed2244ef25d1 service swift] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:04:01.328213 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729c9e2d-f013-4c7e-9001-ed2244ef25d1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:04:01.340535 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729c9e2d-f013-4c7e-9001-ed2244ef25d1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:04:01.342200 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729c9e2d-f013-4c7e-9001-ed2244ef25d1 service swift] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:04:01.351126 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1168/2306] 10.222.0.46 () {60 vars in 1384 bytes} [Thu Jun 23 16:04:01 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 23 16:04:07.276040 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73df4230-bd03-4b7a-bb42-6822dc52b212 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:04:07.277561 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-73df4230-bd03-4b7a-bb42-6822dc52b212 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:04:07.420372 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73df4230-bd03-4b7a-bb42-6822dc52b212 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:04:07.426113 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa032b6-e223-401e-8924-a074979a0c22 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f66988b73644fe38b9849663ee70f25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:04:07.426958 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa032b6-e223-401e-8924-a074979a0c22 service placement] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:04:07.427308 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa032b6-e223-401e-8924-a074979a0c22 service placement] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:04:07.427657 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa032b6-e223-401e-8924-a074979a0c22 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:04:07.429706 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-efa032b6-e223-401e-8924-a074979a0c22 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:04:07.445548 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa032b6-e223-401e-8924-a074979a0c22 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:04:07.447835 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa032b6-e223-401e-8924-a074979a0c22 service placement] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:04:07.459197 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1139/2307] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jun 23 16:04:07 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 16:04:11.347573 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=32 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:04:17.454297 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=73 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:04:25.576127 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd41ebdc-d1eb-47ec-b3a7-8dff13557ea6 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:04:25.576127 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd41ebdc-d1eb-47ec-b3a7-8dff13557ea6 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:04:25.576127 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd41ebdc-d1eb-47ec-b3a7-8dff13557ea6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:04:25.626644 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-bd41ebdc-d1eb-47ec-b3a7-8dff13557ea6 None None] MFA Rules not processed for user `be48addfaa154555b8d0cb7aac8b32ac`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:04:25.627926 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-bd41ebdc-d1eb-47ec-b3a7-8dff13557ea6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:04:25.645769 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1169/2308] 10.222.0.46 () {58 vars in 967 bytes} [Thu Jun 23 16:04:25 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 16:04:25.659266 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa032b6-e223-401e-8924-a074979a0c22 service placement] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:04:25.681748 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa032b6-e223-401e-8924-a074979a0c22 service placement] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:04:25.687769 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6898407-f74e-4921-86e6-4012554c353f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '242bf02d14724fe4837fcce8730377f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:04:25.688439 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6898407-f74e-4921-86e6-4012554c353f service swift] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:04:25.688720 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6898407-f74e-4921-86e6-4012554c353f service swift] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:04:25.688994 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6898407-f74e-4921-86e6-4012554c353f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:04:25.698787 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6898407-f74e-4921-86e6-4012554c353f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:04:25.700414 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6898407-f74e-4921-86e6-4012554c353f service swift] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:04:25.709794 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1140/2309] 10.222.0.46 () {60 vars in 1384 bytes} [Thu Jun 23 16:04:25 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 23 16:04:27.744689 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd41ebdc-d1eb-47ec-b3a7-8dff13557ea6 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:04:27.746269 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-bd41ebdc-d1eb-47ec-b3a7-8dff13557ea6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:04:27.819021 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd41ebdc-d1eb-47ec-b3a7-8dff13557ea6 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:04:27.826149 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b44897-b5e3-4537-8c62-03b077de735a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:04:27.827172 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b44897-b5e3-4537-8c62-03b077de735a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:04:27.827462 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b44897-b5e3-4537-8c62-03b077de735a admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:04:27.827740 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b44897-b5e3-4537-8c62-03b077de735a admin admin] PATH_INFO: `/v3/users/ad8dac501e254a8d9671083b7b7d413c` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:04:27.829078 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b44897-b5e3-4537-8c62-03b077de735a admin admin] RBAC: Authorizing `identity:delete_user(user_id=ad8dac501e254a8d9671083b7b7d413c)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:04:27.831161 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b44897-b5e3-4537-8c62-03b077de735a admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:04:27.879760 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-e6b44897-b5e3-4537-8c62-03b077de735a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ad8dac501e254a8d9671083b7b7d413c'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:04:27.887754 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-e6b44897-b5e3-4537-8c62-03b077de735a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ad8dac501e254a8d9671083b7b7d413c'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:04:27.888442 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-e6b44897-b5e3-4537-8c62-03b077de735a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ad8dac501e254a8d9671083b7b7d413c'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:04:27.889048 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-e6b44897-b5e3-4537-8c62-03b077de735a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ad8dac501e254a8d9671083b7b7d413c'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:04:27.902274 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-e6b44897-b5e3-4537-8c62-03b077de735a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ad8dac501e254a8d9671083b7b7d413c'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:04:27.916735 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1170/2310] 10.222.0.46 () {58 vars in 1223 bytes} [Thu Jun 23 16:04:27 2022] DELETE /identity/v3/users/ad8dac501e254a8d9671083b7b7d413c => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 16:04:28.093823 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6898407-f74e-4921-86e6-4012554c353f service swift] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:04:28.095511 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-c6898407-f74e-4921-86e6-4012554c353f service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:04:28.111407 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6898407-f74e-4921-86e6-4012554c353f service swift] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:04:28.113246 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b44897-b5e3-4537-8c62-03b077de735a admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:04:28.115565 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-e6b44897-b5e3-4537-8c62-03b077de735a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:04:28.181412 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f96931-0176-4428-9e84-b1de9e763565 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:04:28.182301 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f96931-0176-4428-9e84-b1de9e763565 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:04:28.182594 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f96931-0176-4428-9e84-b1de9e763565 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:04:28.182905 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f96931-0176-4428-9e84-b1de9e763565 admin admin] PATH_INFO: `/v3/users/77432115332c4c12a5fe83cb2f9af633` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:04:28.184496 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f96931-0176-4428-9e84-b1de9e763565 admin admin] RBAC: Authorizing `identity:delete_user(user_id=77432115332c4c12a5fe83cb2f9af633)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:04:28.186906 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f96931-0176-4428-9e84-b1de9e763565 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:04:28.214924 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:04:28.244658 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b44897-b5e3-4537-8c62-03b077de735a admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:04:28.252761 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ad1d67-1688-4f87-bda8-6fb8c8e3df73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:04:28.253657 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ad1d67-1688-4f87-bda8-6fb8c8e3df73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:04:28.253949 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ad1d67-1688-4f87-bda8-6fb8c8e3df73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:04:28.254233 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ad1d67-1688-4f87-bda8-6fb8c8e3df73 admin admin] PATH_INFO: `/v3/projects/ab9c00b7758f45daa7063ee2327c59d0` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:04:28.255546 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ad1d67-1688-4f87-bda8-6fb8c8e3df73 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ab9c00b7758f45daa7063ee2327c59d0)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:04:28.257471 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ad1d67-1688-4f87-bda8-6fb8c8e3df73 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:04:28.263630 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-47f96931-0176-4428-9e84-b1de9e763565 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '77432115332c4c12a5fe83cb2f9af633'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:04:28.264326 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-47f96931-0176-4428-9e84-b1de9e763565 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '77432115332c4c12a5fe83cb2f9af633'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:04:28.271974 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-47f96931-0176-4428-9e84-b1de9e763565 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '77432115332c4c12a5fe83cb2f9af633'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:04:28.279860 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-47f96931-0176-4428-9e84-b1de9e763565 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '77432115332c4c12a5fe83cb2f9af633'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:04:28.280527 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-47f96931-0176-4428-9e84-b1de9e763565 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '77432115332c4c12a5fe83cb2f9af633'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:04:28.299125 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1141/2311] 10.222.0.46 () {58 vars in 1223 bytes} [Thu Jun 23 16:04:28 2022] DELETE /identity/v3/users/77432115332c4c12a5fe83cb2f9af633 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 16:04:28.303486 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fb5ed0-25cb-4186-ac3e-c7624b993cd1 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:04:28.303783 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fb5ed0-25cb-4186-ac3e-c7624b993cd1 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:04:28.304068 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fb5ed0-25cb-4186-ac3e-c7624b993cd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:04:28.309543 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-d2ad1d67-1688-4f87-bda8-6fb8c8e3df73 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ab9c00b7758f45daa7063ee2327c59d0', 'request_id': 'req-d2ad1d67-1688-4f87-bda8-6fb8c8e3df73'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:04:28.315624 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-d2ad1d67-1688-4f87-bda8-6fb8c8e3df73 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ab9c00b7758f45daa7063ee2327c59d0', 'request_id': 'req-d2ad1d67-1688-4f87-bda8-6fb8c8e3df73'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:04:28.324199 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-26fb5ed0-25cb-4186-ac3e-c7624b993cd1 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:04:28.325375 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-d2ad1d67-1688-4f87-bda8-6fb8c8e3df73 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ab9c00b7758f45daa7063ee2327c59d0', 'request_id': 'req-d2ad1d67-1688-4f87-bda8-6fb8c8e3df73'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:04:28.325523 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-26fb5ed0-25cb-4186-ac3e-c7624b993cd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:04:28.345494 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-d2ad1d67-1688-4f87-bda8-6fb8c8e3df73 admin admin] The token cache is being invalidate because project ab9c00b7758f45daa7063ee2327c59d0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80153) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 16:04:28.345875 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-d2ad1d67-1688-4f87-bda8-6fb8c8e3df73 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:04:28.347398 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-d2ad1d67-1688-4f87-bda8-6fb8c8e3df73 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:04:28.350459 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1171/2312] 10.222.0.46 () {58 vars in 1232 bytes} [Thu Jun 23 16:04:28 2022] DELETE /identity/v3/projects/ab9c00b7758f45daa7063ee2327c59d0 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 16:04:28.357574 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b68c342-adcf-4942-b74e-3a530c1f1df7 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:04:28.357876 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b68c342-adcf-4942-b74e-3a530c1f1df7 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:04:28.358156 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b68c342-adcf-4942-b74e-3a530c1f1df7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:04:28.388817 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:04:28.396275 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-5b68c342-adcf-4942-b74e-3a530c1f1df7 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:04:28.397506 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-5b68c342-adcf-4942-b74e-3a530c1f1df7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:04:28.400373 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1142/2313] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:04:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 16:04:28.407223 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1172/2314] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:04:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 16:04:28.410044 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fb5ed0-25cb-4186-ac3e-c7624b993cd1 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:04:28.411421 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-26fb5ed0-25cb-4186-ac3e-c7624b993cd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:04:28.415700 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcc7966-a06b-4ffb-b8e4-678bbb0273bf None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:04:28.415993 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcc7966-a06b-4ffb-b8e4-678bbb0273bf None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:04:28.416278 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcc7966-a06b-4ffb-b8e4-678bbb0273bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:04:28.435516 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-4dcc7966-a06b-4ffb-b8e4-678bbb0273bf None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:04:28.436739 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-4dcc7966-a06b-4ffb-b8e4-678bbb0273bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:04:28.445262 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1173/2315] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:04:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 16:04:28.448964 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcc7966-a06b-4ffb-b8e4-678bbb0273bf None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:04:28.458220 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcc7966-a06b-4ffb-b8e4-678bbb0273bf None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:04:28.463178 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415b6fbd-953a-4988-b7c1-f7463a3cb4d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:04:28.463853 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415b6fbd-953a-4988-b7c1-f7463a3cb4d6 admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:04:28.464146 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415b6fbd-953a-4988-b7c1-f7463a3cb4d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:04:28.464421 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415b6fbd-953a-4988-b7c1-f7463a3cb4d6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:04:28.465157 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415b6fbd-953a-4988-b7c1-f7463a3cb4d6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 16:04:28.465470 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415b6fbd-953a-4988-b7c1-f7463a3cb4d6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:04:28.467131 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415b6fbd-953a-4988-b7c1-f7463a3cb4d6 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:04:28.474373 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fb5ed0-25cb-4186-ac3e-c7624b993cd1 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:04:28.478736 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e4a3aa-a8ac-4c26-ae43-d5752446a88c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51784064951a4619ae09b9167fb0fd0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:04:28.479487 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e4a3aa-a8ac-4c26-ae43-d5752446a88c service neutron] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:04:28.479973 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e4a3aa-a8ac-4c26-ae43-d5752446a88c service neutron] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:04:28.480092 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1174/2316] 10.222.0.46 () {58 vars in 1152 bytes} [Thu Jun 23 16:04:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 16:04:28.480204 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e4a3aa-a8ac-4c26-ae43-d5752446a88c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:04:28.484336 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415b6fbd-953a-4988-b7c1-f7463a3cb4d6 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:04:28.489682 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e4a3aa-a8ac-4c26-ae43-d5752446a88c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:04:28.491300 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e4a3aa-a8ac-4c26-ae43-d5752446a88c service neutron] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:04:28.494472 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415b6fbd-953a-4988-b7c1-f7463a3cb4d6 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:04:28.500043 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b74fd1-bf71-4d44-8d6b-5a12f84746be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:04:28.500702 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b74fd1-bf71-4d44-8d6b-5a12f84746be admin admin] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:04:28.500992 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b74fd1-bf71-4d44-8d6b-5a12f84746be admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:04:28.501280 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b74fd1-bf71-4d44-8d6b-5a12f84746be admin admin] PATH_INFO: `/v3/projects` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:04:28.501975 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b74fd1-bf71-4d44-8d6b-5a12f84746be admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:04:28.502870 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1143/2317] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jun 23 16:04:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 16:04:28.503935 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b74fd1-bf71-4d44-8d6b-5a12f84746be admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:04:28.532039 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1175/2318] 10.222.0.46 () {60 vars in 1152 bytes} [Thu Jun 23 16:04:28 2022] POST /identity/v3/projects => generated 377 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 16:04:28.536305 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e4a3aa-a8ac-4c26-ae43-d5752446a88c service neutron] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:04:28.546965 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e4a3aa-a8ac-4c26-ae43-d5752446a88c service neutron] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:04:28.622233 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5a0308-6b21-4980-ba33-fd49662db9f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:04:28.622975 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5a0308-6b21-4980-ba33-fd49662db9f2 admin admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:04:28.623264 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5a0308-6b21-4980-ba33-fd49662db9f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:04:28.623542 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5a0308-6b21-4980-ba33-fd49662db9f2 admin admin] PATH_INFO: `/v3/users` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:04:28.624245 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5a0308-6b21-4980-ba33-fd49662db9f2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:04:28.626510 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5a0308-6b21-4980-ba33-fd49662db9f2 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:04:28.651231 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1144/2319] 10.222.0.46 () {60 vars in 1143 bytes} [Thu Jun 23 16:04:28 2022] POST /identity/v3/users => generated 289 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 16:04:28.652590 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b74fd1-bf71-4d44-8d6b-5a12f84746be admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:04:28.664465 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b74fd1-bf71-4d44-8d6b-5a12f84746be admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:04:28.671503 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c744750a-775b-40a4-8671-9112a4bb57bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:04:28.672192 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c744750a-775b-40a4-8671-9112a4bb57bf admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:04:28.672480 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c744750a-775b-40a4-8671-9112a4bb57bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:04:28.672760 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c744750a-775b-40a4-8671-9112a4bb57bf admin admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:04:28.673503 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c744750a-775b-40a4-8671-9112a4bb57bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:04:28.675239 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c744750a-775b-40a4-8671-9112a4bb57bf admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:04:28.686799 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1176/2320] 10.222.0.46 () {58 vars in 1121 bytes} [Thu Jun 23 16:04:28 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 16:04:28.691114 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5a0308-6b21-4980-ba33-fd49662db9f2 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:04:28.701511 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5a0308-6b21-4980-ba33-fd49662db9f2 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:04:28.706570 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0a2d6f-9f5e-4965-a685-cc07ccde49db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:04:28.707279 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0a2d6f-9f5e-4965-a685-cc07ccde49db admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:04:28.707568 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0a2d6f-9f5e-4965-a685-cc07ccde49db admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:04:28.707979 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0a2d6f-9f5e-4965-a685-cc07ccde49db admin admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:04:28.708729 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0a2d6f-9f5e-4965-a685-cc07ccde49db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:04:28.710374 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0a2d6f-9f5e-4965-a685-cc07ccde49db admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:04:28.724934 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c744750a-775b-40a4-8671-9112a4bb57bf admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:04:28.725950 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1145/2321] 10.222.0.46 () {58 vars in 1121 bytes} [Thu Jun 23 16:04:28 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 16:04:28.735743 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c744750a-775b-40a4-8671-9112a4bb57bf admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:04:28.747406 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdd2830-17a8-4836-8fef-e1b708788a36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:04:28.749072 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdd2830-17a8-4836-8fef-e1b708788a36 admin admin] REQUEST_METHOD: `PUT` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:04:28.749433 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdd2830-17a8-4836-8fef-e1b708788a36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:04:28.749784 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdd2830-17a8-4836-8fef-e1b708788a36 admin admin] PATH_INFO: `/v3/projects/42ebe5dc582e4a6b92d4f18002a0ffba/users/c953617a01f1401c966c16467138d628/roles/e2a5bbf3200a4d4091d72bc7ff78c34d` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:04:28.773859 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdd2830-17a8-4836-8fef-e1b708788a36 admin admin] RBAC: Authorizing `identity:create_grant(project_id=42ebe5dc582e4a6b92d4f18002a0ffba, user_id=c953617a01f1401c966c16467138d628, role_id=e2a5bbf3200a4d4091d72bc7ff78c34d)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:04:28.778541 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdd2830-17a8-4836-8fef-e1b708788a36 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:04:28.794094 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0a2d6f-9f5e-4965-a685-cc07ccde49db admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:04:28.795919 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-7b0a2d6f-9f5e-4965-a685-cc07ccde49db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:04:28.799125 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1177/2322] 10.222.0.46 () {60 vars in 1482 bytes} [Thu Jun 23 16:04:28 2022] PUT /identity/v3/projects/42ebe5dc582e4a6b92d4f18002a0ffba/users/c953617a01f1401c966c16467138d628/roles/e2a5bbf3200a4d4091d72bc7ff78c34d => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 16:04:28.805696 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0e6c21-d564-4dae-9387-6440a467cc6e None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:04:28.805982 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0e6c21-d564-4dae-9387-6440a467cc6e None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:04:28.806267 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0e6c21-d564-4dae-9387-6440a467cc6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:04:28.817244 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0a2d6f-9f5e-4965-a685-cc07ccde49db admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:04:28.829478 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-1e0e6c21-d564-4dae-9387-6440a467cc6e None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:04:28.832780 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-1e0e6c21-d564-4dae-9387-6440a467cc6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:04:28.873371 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:04:28.912863 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04265d1-8058-4ad1-bb15-56017acf2cf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:04:28.913862 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04265d1-8058-4ad1-bb15-56017acf2cf6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:04:28.914742 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04265d1-8058-4ad1-bb15-56017acf2cf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:04:28.915096 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04265d1-8058-4ad1-bb15-56017acf2cf6 admin admin] PATH_INFO: `/v3/projects/29056405294245a2b4a149bde8bf16da` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:04:28.916209 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04265d1-8058-4ad1-bb15-56017acf2cf6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=29056405294245a2b4a149bde8bf16da)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:04:28.916501 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1178/2323] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:04:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 16:04:28.918120 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04265d1-8058-4ad1-bb15-56017acf2cf6 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:04:28.926048 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0e6c21-d564-4dae-9387-6440a467cc6e None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:04:28.943369 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:04:28.970838 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-d04265d1-8058-4ad1-bb15-56017acf2cf6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '29056405294245a2b4a149bde8bf16da', 'request_id': 'req-d04265d1-8058-4ad1-bb15-56017acf2cf6'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:04:28.977740 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-d04265d1-8058-4ad1-bb15-56017acf2cf6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '29056405294245a2b4a149bde8bf16da', 'request_id': 'req-d04265d1-8058-4ad1-bb15-56017acf2cf6'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:04:28.984445 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0e6c21-d564-4dae-9387-6440a467cc6e None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:04:28.985609 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-d04265d1-8058-4ad1-bb15-56017acf2cf6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '29056405294245a2b4a149bde8bf16da', 'request_id': 'req-d04265d1-8058-4ad1-bb15-56017acf2cf6'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:04:28.997625 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-d04265d1-8058-4ad1-bb15-56017acf2cf6 admin admin] The token cache is being invalidate because project 29056405294245a2b4a149bde8bf16da was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80154) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 16:04:28.997920 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-d04265d1-8058-4ad1-bb15-56017acf2cf6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:04:28.998580 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-d04265d1-8058-4ad1-bb15-56017acf2cf6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:04:29.000217 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1146/2324] 10.222.0.46 () {58 vars in 1232 bytes} [Thu Jun 23 16:04:28 2022] DELETE /identity/v3/projects/29056405294245a2b4a149bde8bf16da => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 16:04:29.040217 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21864bf9-4ae4-41a3-9bb1-b306870924ac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51784064951a4619ae09b9167fb0fd0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:04:29.040217 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21864bf9-4ae4-41a3-9bb1-b306870924ac service neutron] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:04:29.040438 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21864bf9-4ae4-41a3-9bb1-b306870924ac service neutron] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:04:29.040714 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21864bf9-4ae4-41a3-9bb1-b306870924ac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:04:29.042834 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-21864bf9-4ae4-41a3-9bb1-b306870924ac service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:04:29.116931 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21864bf9-4ae4-41a3-9bb1-b306870924ac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:04:29.118550 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21864bf9-4ae4-41a3-9bb1-b306870924ac service neutron] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:04:29.129938 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1179/2325] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jun 23 16:04:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 16:04:29.288086 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04265d1-8058-4ad1-bb15-56017acf2cf6 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:04:29.289960 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-d04265d1-8058-4ad1-bb15-56017acf2cf6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:04:29.302765 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04265d1-8058-4ad1-bb15-56017acf2cf6 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:04:29.308808 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fbcb53-58ce-4c12-bb8b-95e2b8616b78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:04:29.309878 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fbcb53-58ce-4c12-bb8b-95e2b8616b78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:04:29.310279 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fbcb53-58ce-4c12-bb8b-95e2b8616b78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:04:29.310622 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fbcb53-58ce-4c12-bb8b-95e2b8616b78 admin admin] PATH_INFO: `/v3/users/e805623e339a46db99b7040fd2a6069e` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:04:29.312072 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fbcb53-58ce-4c12-bb8b-95e2b8616b78 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e805623e339a46db99b7040fd2a6069e)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:04:29.314399 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fbcb53-58ce-4c12-bb8b-95e2b8616b78 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:04:29.396951 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-51fbcb53-58ce-4c12-bb8b-95e2b8616b78 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e805623e339a46db99b7040fd2a6069e'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:04:29.397646 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-51fbcb53-58ce-4c12-bb8b-95e2b8616b78 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e805623e339a46db99b7040fd2a6069e'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:04:29.405772 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-51fbcb53-58ce-4c12-bb8b-95e2b8616b78 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e805623e339a46db99b7040fd2a6069e'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:04:29.414458 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-51fbcb53-58ce-4c12-bb8b-95e2b8616b78 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e805623e339a46db99b7040fd2a6069e'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:04:29.415331 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-51fbcb53-58ce-4c12-bb8b-95e2b8616b78 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e805623e339a46db99b7040fd2a6069e'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:04:29.437209 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1147/2326] 10.222.0.46 () {58 vars in 1223 bytes} [Thu Jun 23 16:04:29 2022] DELETE /identity/v3/users/e805623e339a46db99b7040fd2a6069e => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 16:04:29.441523 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfadd0e-f745-4475-b1a4-2359eb210ecd None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:04:29.441878 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfadd0e-f745-4475-b1a4-2359eb210ecd None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:04:29.442224 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfadd0e-f745-4475-b1a4-2359eb210ecd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:04:29.460486 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-ebfadd0e-f745-4475-b1a4-2359eb210ecd None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:04:29.462465 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-ebfadd0e-f745-4475-b1a4-2359eb210ecd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:04:29.516718 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:04:29.543919 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1180/2327] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:04:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 16:04:29.558379 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fbcb53-58ce-4c12-bb8b-95e2b8616b78 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:04:29.559922 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-51fbcb53-58ce-4c12-bb8b-95e2b8616b78 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:04:29.622310 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fbcb53-58ce-4c12-bb8b-95e2b8616b78 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:04:29.625987 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abd5677-df40-4c53-98ff-f78c6aba7b28 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51784064951a4619ae09b9167fb0fd0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:04:29.626662 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abd5677-df40-4c53-98ff-f78c6aba7b28 service neutron] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:04:29.626976 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abd5677-df40-4c53-98ff-f78c6aba7b28 service neutron] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:04:29.627255 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abd5677-df40-4c53-98ff-f78c6aba7b28 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:04:29.637167 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abd5677-df40-4c53-98ff-f78c6aba7b28 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:04:29.638601 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abd5677-df40-4c53-98ff-f78c6aba7b28 service neutron] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:04:29.649320 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1148/2328] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jun 23 16:04:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 16:04:30.250197 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfadd0e-f745-4475-b1a4-2359eb210ecd None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:04:30.251918 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-ebfadd0e-f745-4475-b1a4-2359eb210ecd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:04:30.272533 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfadd0e-f745-4475-b1a4-2359eb210ecd None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:04:30.283166 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd62e42-c9c0-4c60-8b6e-b180a4f10e01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:04:30.284175 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd62e42-c9c0-4c60-8b6e-b180a4f10e01 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:04:30.284684 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd62e42-c9c0-4c60-8b6e-b180a4f10e01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:04:30.285617 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd62e42-c9c0-4c60-8b6e-b180a4f10e01 admin admin] PATH_INFO: `/v3/projects/9710e33d284d4851b5b7cc47b09546df` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:04:30.287147 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd62e42-c9c0-4c60-8b6e-b180a4f10e01 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9710e33d284d4851b5b7cc47b09546df)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:04:30.290422 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd62e42-c9c0-4c60-8b6e-b180a4f10e01 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:04:30.353162 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-acd62e42-c9c0-4c60-8b6e-b180a4f10e01 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9710e33d284d4851b5b7cc47b09546df', 'request_id': 'req-acd62e42-c9c0-4c60-8b6e-b180a4f10e01'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:04:30.361369 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-acd62e42-c9c0-4c60-8b6e-b180a4f10e01 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9710e33d284d4851b5b7cc47b09546df', 'request_id': 'req-acd62e42-c9c0-4c60-8b6e-b180a4f10e01'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:04:30.367945 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-acd62e42-c9c0-4c60-8b6e-b180a4f10e01 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9710e33d284d4851b5b7cc47b09546df', 'request_id': 'req-acd62e42-c9c0-4c60-8b6e-b180a4f10e01'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:04:30.384061 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-acd62e42-c9c0-4c60-8b6e-b180a4f10e01 admin admin] The token cache is being invalidate because project 9710e33d284d4851b5b7cc47b09546df was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80153) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 16:04:30.384328 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-acd62e42-c9c0-4c60-8b6e-b180a4f10e01 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:04:30.385553 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-acd62e42-c9c0-4c60-8b6e-b180a4f10e01 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:04:30.388249 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1181/2329] 10.222.0.46 () {58 vars in 1232 bytes} [Thu Jun 23 16:04:30 2022] DELETE /identity/v3/projects/9710e33d284d4851b5b7cc47b09546df => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 16:04:30.393506 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d781a389-1952-4a61-a0da-6eef723a0cb6 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:04:30.394027 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d781a389-1952-4a61-a0da-6eef723a0cb6 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:04:30.394125 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d781a389-1952-4a61-a0da-6eef723a0cb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:04:30.416894 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-d781a389-1952-4a61-a0da-6eef723a0cb6 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:04:30.418792 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-d781a389-1952-4a61-a0da-6eef723a0cb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:04:30.441242 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:DELETE=17,keystone:SELECT=81,keystone:INSERT=2 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:04:30.509960 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1149/2330] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:04:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 16:04:30.518859 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95363953-1369-4325-8b1d-a3461da4a621 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:04:30.519175 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95363953-1369-4325-8b1d-a3461da4a621 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:04:30.520012 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95363953-1369-4325-8b1d-a3461da4a621 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:04:30.540708 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-95363953-1369-4325-8b1d-a3461da4a621 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:04:30.542069 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-95363953-1369-4325-8b1d-a3461da4a621 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:04:30.555615 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1182/2331] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:04:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 16:04:30.556660 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d781a389-1952-4a61-a0da-6eef723a0cb6 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:04:30.571569 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d781a389-1952-4a61-a0da-6eef723a0cb6 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:04:30.576849 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc77d08-3ebd-427f-9829-37f241dce608 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:04:30.577912 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc77d08-3ebd-427f-9829-37f241dce608 admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:04:30.579061 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc77d08-3ebd-427f-9829-37f241dce608 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:04:30.579343 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc77d08-3ebd-427f-9829-37f241dce608 admin admin] PATH_INFO: `/v3/domains` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:04:30.580098 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc77d08-3ebd-427f-9829-37f241dce608 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 16:04:30.580418 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc77d08-3ebd-427f-9829-37f241dce608 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:04:30.582130 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc77d08-3ebd-427f-9829-37f241dce608 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:04:30.595543 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1150/2332] 10.222.0.46 () {58 vars in 1152 bytes} [Thu Jun 23 16:04:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 16:04:30.600613 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95363953-1369-4325-8b1d-a3461da4a621 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:04:30.611435 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95363953-1369-4325-8b1d-a3461da4a621 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:04:30.616234 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe15d5d-f249-4ac1-9d32-54da4a2744d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:04:30.616910 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe15d5d-f249-4ac1-9d32-54da4a2744d7 admin admin] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:04:30.617232 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe15d5d-f249-4ac1-9d32-54da4a2744d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:04:30.617496 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe15d5d-f249-4ac1-9d32-54da4a2744d7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:04:30.618459 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe15d5d-f249-4ac1-9d32-54da4a2744d7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:04:30.621156 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe15d5d-f249-4ac1-9d32-54da4a2744d7 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:04:30.678729 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1183/2333] 10.222.0.46 () {60 vars in 1152 bytes} [Thu Jun 23 16:04:30 2022] POST /identity/v3/projects => generated 373 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 16:04:30.682261 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc77d08-3ebd-427f-9829-37f241dce608 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:04:30.692612 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc77d08-3ebd-427f-9829-37f241dce608 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:04:30.778753 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0b4c44-2c60-4971-8c52-cf605d103056 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:04:30.779658 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0b4c44-2c60-4971-8c52-cf605d103056 admin admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:04:30.779938 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0b4c44-2c60-4971-8c52-cf605d103056 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:04:30.780378 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0b4c44-2c60-4971-8c52-cf605d103056 admin admin] PATH_INFO: `/v3/users` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:04:30.781901 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0b4c44-2c60-4971-8c52-cf605d103056 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:04:30.783494 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0b4c44-2c60-4971-8c52-cf605d103056 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:04:30.809736 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1151/2334] 10.222.0.46 () {60 vars in 1143 bytes} [Thu Jun 23 16:04:30 2022] POST /identity/v3/users => generated 287 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 16:04:30.811547 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe15d5d-f249-4ac1-9d32-54da4a2744d7 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:04:30.826814 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe15d5d-f249-4ac1-9d32-54da4a2744d7 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:04:30.833523 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf36e50d-8a5b-4fc9-b153-e34407c180ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:04:30.834217 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf36e50d-8a5b-4fc9-b153-e34407c180ad admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:04:30.834502 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf36e50d-8a5b-4fc9-b153-e34407c180ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:04:30.834811 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf36e50d-8a5b-4fc9-b153-e34407c180ad admin admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:04:30.835558 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf36e50d-8a5b-4fc9-b153-e34407c180ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:04:30.837772 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf36e50d-8a5b-4fc9-b153-e34407c180ad admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:04:30.849409 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1184/2335] 10.222.0.46 () {58 vars in 1121 bytes} [Thu Jun 23 16:04:30 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 16:04:30.853806 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0b4c44-2c60-4971-8c52-cf605d103056 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:04:30.865032 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0b4c44-2c60-4971-8c52-cf605d103056 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:04:30.869909 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0eac3f1-71d1-475b-bd87-683798c590f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:04:30.870588 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0eac3f1-71d1-475b-bd87-683798c590f9 admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:04:30.870911 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0eac3f1-71d1-475b-bd87-683798c590f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:04:30.871191 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0eac3f1-71d1-475b-bd87-683798c590f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:04:30.871936 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0eac3f1-71d1-475b-bd87-683798c590f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:04:30.873550 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0eac3f1-71d1-475b-bd87-683798c590f9 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:04:30.884432 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1152/2336] 10.222.0.46 () {58 vars in 1121 bytes} [Thu Jun 23 16:04:30 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 16:04:30.888696 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf36e50d-8a5b-4fc9-b153-e34407c180ad admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:04:30.900411 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf36e50d-8a5b-4fc9-b153-e34407c180ad admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:04:30.906547 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6921b1b1-3c1d-493b-a6f6-048437a6c76b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:04:30.907479 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6921b1b1-3c1d-493b-a6f6-048437a6c76b admin admin] REQUEST_METHOD: `PUT` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:04:30.907759 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6921b1b1-3c1d-493b-a6f6-048437a6c76b admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:04:30.908036 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6921b1b1-3c1d-493b-a6f6-048437a6c76b admin admin] PATH_INFO: `/v3/projects/ff2e0b9bc8dd474eb6704d5d8309e3a2/users/8701107e43cc467a88b1e24e47c100bb/roles/e2a5bbf3200a4d4091d72bc7ff78c34d` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:04:30.933785 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6921b1b1-3c1d-493b-a6f6-048437a6c76b admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff2e0b9bc8dd474eb6704d5d8309e3a2, user_id=8701107e43cc467a88b1e24e47c100bb, role_id=e2a5bbf3200a4d4091d72bc7ff78c34d)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:04:30.936682 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6921b1b1-3c1d-493b-a6f6-048437a6c76b admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:04:30.959834 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1185/2337] 10.222.0.46 () {60 vars in 1482 bytes} [Thu Jun 23 16:04:30 2022] PUT /identity/v3/projects/ff2e0b9bc8dd474eb6704d5d8309e3a2/users/8701107e43cc467a88b1e24e47c100bb/roles/e2a5bbf3200a4d4091d72bc7ff78c34d => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 16:04:30.977739 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316e6fc3-c6c7-4cba-a687-338e27c9d035 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:04:30.978023 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316e6fc3-c6c7-4cba-a687-338e27c9d035 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:04:30.978359 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316e6fc3-c6c7-4cba-a687-338e27c9d035 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:04:31.012281 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-316e6fc3-c6c7-4cba-a687-338e27c9d035 None None] MFA Rules not processed for user `8701107e43cc467a88b1e24e47c100bb`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:04:31.014290 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-316e6fc3-c6c7-4cba-a687-338e27c9d035 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:04:31.090215 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1153/2338] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:04:30 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 16:04:31.102231 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6921b1b1-3c1d-493b-a6f6-048437a6c76b admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:04:31.104186 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-6921b1b1-3c1d-493b-a6f6-048437a6c76b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:04:31.189841 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:04:31.203288 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6921b1b1-3c1d-493b-a6f6-048437a6c76b admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:04:31.211376 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6b8aa6-0c57-4b71-8525-e2151e0d6a92 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce18eaaba3a6477684fff1fd98ec8b51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:04:31.211376 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6b8aa6-0c57-4b71-8525-e2151e0d6a92 service cinder] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:04:31.211376 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6b8aa6-0c57-4b71-8525-e2151e0d6a92 service cinder] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:04:31.211889 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6b8aa6-0c57-4b71-8525-e2151e0d6a92 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:04:31.221987 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6b8aa6-0c57-4b71-8525-e2151e0d6a92 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:04:31.223830 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6b8aa6-0c57-4b71-8525-e2151e0d6a92 service cinder] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:04:31.235267 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1186/2339] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jun 23 16:04:31 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 16:04:31.264419 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316e6fc3-c6c7-4cba-a687-338e27c9d035 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:04:31.289866 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:04:31.354109 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316e6fc3-c6c7-4cba-a687-338e27c9d035 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:04:31.361523 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9864acec-de68-4658-9e73-3efaecdc22bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:04:31.363481 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9864acec-de68-4658-9e73-3efaecdc22bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:04:31.363847 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9864acec-de68-4658-9e73-3efaecdc22bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:04:31.364200 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9864acec-de68-4658-9e73-3efaecdc22bf admin admin] PATH_INFO: `/v3/users/8701107e43cc467a88b1e24e47c100bb` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:04:31.366770 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9864acec-de68-4658-9e73-3efaecdc22bf admin admin] RBAC: Authorizing `identity:delete_user(user_id=8701107e43cc467a88b1e24e47c100bb)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:04:31.370493 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9864acec-de68-4658-9e73-3efaecdc22bf admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:04:31.445096 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-9864acec-de68-4658-9e73-3efaecdc22bf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8701107e43cc467a88b1e24e47c100bb'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:04:31.446112 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-9864acec-de68-4658-9e73-3efaecdc22bf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8701107e43cc467a88b1e24e47c100bb'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:04:31.455975 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-9864acec-de68-4658-9e73-3efaecdc22bf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8701107e43cc467a88b1e24e47c100bb'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:04:31.465121 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-9864acec-de68-4658-9e73-3efaecdc22bf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8701107e43cc467a88b1e24e47c100bb'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:04:31.466335 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-9864acec-de68-4658-9e73-3efaecdc22bf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8701107e43cc467a88b1e24e47c100bb'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:04:31.490471 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1154/2340] 10.222.0.46 () {58 vars in 1223 bytes} [Thu Jun 23 16:04:31 2022] DELETE /identity/v3/users/8701107e43cc467a88b1e24e47c100bb => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 16:04:31.495756 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807d4fb7-9495-4d66-aee7-0200a52f9e6a None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:04:31.496259 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807d4fb7-9495-4d66-aee7-0200a52f9e6a None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:04:31.496458 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807d4fb7-9495-4d66-aee7-0200a52f9e6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:04:31.518520 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-807d4fb7-9495-4d66-aee7-0200a52f9e6a None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:04:31.519933 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-807d4fb7-9495-4d66-aee7-0200a52f9e6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:04:31.603273 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1187/2341] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:04:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 16:04:31.612963 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9864acec-de68-4658-9e73-3efaecdc22bf admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:04:31.614499 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-9864acec-de68-4658-9e73-3efaecdc22bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:04:31.683369 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9864acec-de68-4658-9e73-3efaecdc22bf admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:04:31.687318 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee1bd48-62cc-4fa1-8aa0-e21a19651356 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51784064951a4619ae09b9167fb0fd0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:04:31.687977 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee1bd48-62cc-4fa1-8aa0-e21a19651356 service neutron] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:04:31.688260 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee1bd48-62cc-4fa1-8aa0-e21a19651356 service neutron] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:04:31.688534 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee1bd48-62cc-4fa1-8aa0-e21a19651356 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:04:31.698853 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee1bd48-62cc-4fa1-8aa0-e21a19651356 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:04:31.700455 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee1bd48-62cc-4fa1-8aa0-e21a19651356 service neutron] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:04:31.712193 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1155/2342] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jun 23 16:04:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 16:04:32.076385 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807d4fb7-9495-4d66-aee7-0200a52f9e6a None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:04:32.080975 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-807d4fb7-9495-4d66-aee7-0200a52f9e6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:04:32.103084 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807d4fb7-9495-4d66-aee7-0200a52f9e6a None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:04:32.109176 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc21805d-6068-4d18-bf4e-9b87828d07d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:04:32.111269 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc21805d-6068-4d18-bf4e-9b87828d07d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:04:32.111562 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc21805d-6068-4d18-bf4e-9b87828d07d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:04:32.112284 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc21805d-6068-4d18-bf4e-9b87828d07d0 admin admin] PATH_INFO: `/v3/projects/ff2e0b9bc8dd474eb6704d5d8309e3a2` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:04:32.114946 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc21805d-6068-4d18-bf4e-9b87828d07d0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ff2e0b9bc8dd474eb6704d5d8309e3a2)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:04:32.117823 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc21805d-6068-4d18-bf4e-9b87828d07d0 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:04:32.184346 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-fc21805d-6068-4d18-bf4e-9b87828d07d0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ff2e0b9bc8dd474eb6704d5d8309e3a2', 'request_id': 'req-fc21805d-6068-4d18-bf4e-9b87828d07d0'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:04:32.191386 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-fc21805d-6068-4d18-bf4e-9b87828d07d0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ff2e0b9bc8dd474eb6704d5d8309e3a2', 'request_id': 'req-fc21805d-6068-4d18-bf4e-9b87828d07d0'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:04:32.207957 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-fc21805d-6068-4d18-bf4e-9b87828d07d0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ff2e0b9bc8dd474eb6704d5d8309e3a2', 'request_id': 'req-fc21805d-6068-4d18-bf4e-9b87828d07d0'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:04:32.226149 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-fc21805d-6068-4d18-bf4e-9b87828d07d0 admin admin] The token cache is being invalidate because project ff2e0b9bc8dd474eb6704d5d8309e3a2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80153) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 16:04:32.226480 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-fc21805d-6068-4d18-bf4e-9b87828d07d0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:04:32.227166 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-fc21805d-6068-4d18-bf4e-9b87828d07d0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:04:32.229015 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1188/2343] 10.222.0.46 () {58 vars in 1232 bytes} [Thu Jun 23 16:04:32 2022] DELETE /identity/v3/projects/ff2e0b9bc8dd474eb6704d5d8309e3a2 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 16:04:33.536522 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c944e4-1772-4e37-bf4b-7eaaed746f8d None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:04:33.537082 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c944e4-1772-4e37-bf4b-7eaaed746f8d None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:04:33.537082 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c944e4-1772-4e37-bf4b-7eaaed746f8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:04:33.570922 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-68c944e4-1772-4e37-bf4b-7eaaed746f8d None None] MFA Rules not processed for user `c953617a01f1401c966c16467138d628`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:04:33.572175 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-68c944e4-1772-4e37-bf4b-7eaaed746f8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:04:33.606901 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:04:33.644509 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1156/2344] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:04:33 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 16:04:33.649853 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc21805d-6068-4d18-bf4e-9b87828d07d0 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:04:33.651392 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-fc21805d-6068-4d18-bf4e-9b87828d07d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:04:33.736041 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc21805d-6068-4d18-bf4e-9b87828d07d0 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:04:33.742100 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d83c4dd-fcde-4489-9259-983fcbb206ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:04:33.743701 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d83c4dd-fcde-4489-9259-983fcbb206ca admin admin] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:04:33.743805 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d83c4dd-fcde-4489-9259-983fcbb206ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:04:33.744047 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d83c4dd-fcde-4489-9259-983fcbb206ca admin admin] PATH_INFO: `/v3/projects` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:04:33.744886 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d83c4dd-fcde-4489-9259-983fcbb206ca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:04:33.746965 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d83c4dd-fcde-4489-9259-983fcbb206ca admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:04:33.763959 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:04:33.781132 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1189/2345] 10.222.0.46 () {60 vars in 1152 bytes} [Thu Jun 23 16:04:33 2022] POST /identity/v3/projects => generated 379 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 16:04:33.785601 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c944e4-1772-4e37-bf4b-7eaaed746f8d None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:04:33.803191 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c944e4-1772-4e37-bf4b-7eaaed746f8d None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:04:33.879254 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efc586f-6ab7-4e9d-bddd-51e93e964cc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:04:33.879952 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efc586f-6ab7-4e9d-bddd-51e93e964cc5 admin admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:04:33.882228 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efc586f-6ab7-4e9d-bddd-51e93e964cc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:04:33.885597 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efc586f-6ab7-4e9d-bddd-51e93e964cc5 admin admin] PATH_INFO: `/v3/users` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:04:33.886730 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efc586f-6ab7-4e9d-bddd-51e93e964cc5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:04:33.888496 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efc586f-6ab7-4e9d-bddd-51e93e964cc5 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:04:33.911410 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1157/2346] 10.222.0.46 () {60 vars in 1143 bytes} [Thu Jun 23 16:04:33 2022] POST /identity/v3/users => generated 296 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 16:04:33.915401 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d83c4dd-fcde-4489-9259-983fcbb206ca admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:04:33.927515 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d83c4dd-fcde-4489-9259-983fcbb206ca admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:04:33.932406 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2dfff51-fa27-4923-bf71-416c6582f71e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:04:33.933121 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dfff51-fa27-4923-bf71-416c6582f71e admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:04:33.933404 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dfff51-fa27-4923-bf71-416c6582f71e admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:04:33.933680 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dfff51-fa27-4923-bf71-416c6582f71e admin admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:04:33.934419 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2dfff51-fa27-4923-bf71-416c6582f71e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:04:33.936084 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2dfff51-fa27-4923-bf71-416c6582f71e admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:04:33.948205 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1190/2347] 10.222.0.46 () {58 vars in 1121 bytes} [Thu Jun 23 16:04:33 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 16:04:33.952656 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efc586f-6ab7-4e9d-bddd-51e93e964cc5 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:04:33.967770 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efc586f-6ab7-4e9d-bddd-51e93e964cc5 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:04:33.972576 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ad5f8f-47ff-4122-baa6-ff4ea5337e7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:04:33.973518 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ad5f8f-47ff-4122-baa6-ff4ea5337e7a admin admin] REQUEST_METHOD: `PUT` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:04:33.973854 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ad5f8f-47ff-4122-baa6-ff4ea5337e7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:04:33.974139 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ad5f8f-47ff-4122-baa6-ff4ea5337e7a admin admin] PATH_INFO: `/v3/projects/46a39c4592f044408de7c6414c77f7f9/users/fd6d74aa8bf244d0b37f34e718e4f132/roles/812d3c567c52469cb2b35709fbdc857c` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:04:33.991438 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ad5f8f-47ff-4122-baa6-ff4ea5337e7a admin admin] RBAC: Authorizing `identity:create_grant(project_id=46a39c4592f044408de7c6414c77f7f9, user_id=fd6d74aa8bf244d0b37f34e718e4f132, role_id=812d3c567c52469cb2b35709fbdc857c)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:04:33.994673 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ad5f8f-47ff-4122-baa6-ff4ea5337e7a admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:04:34.008782 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1158/2348] 10.222.0.46 () {60 vars in 1482 bytes} [Thu Jun 23 16:04:33 2022] PUT /identity/v3/projects/46a39c4592f044408de7c6414c77f7f9/users/fd6d74aa8bf244d0b37f34e718e4f132/roles/812d3c567c52469cb2b35709fbdc857c => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 16:04:34.012675 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2dfff51-fa27-4923-bf71-416c6582f71e admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:04:34.024231 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2dfff51-fa27-4923-bf71-416c6582f71e admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:04:34.084979 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a867bb-feeb-4808-bf73-aaccdcd64272 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:04:34.086323 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a867bb-feeb-4808-bf73-aaccdcd64272 admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:04:34.086681 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a867bb-feeb-4808-bf73-aaccdcd64272 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:04:34.087054 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a867bb-feeb-4808-bf73-aaccdcd64272 admin admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:04:34.087809 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a867bb-feeb-4808-bf73-aaccdcd64272 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:04:34.089416 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a867bb-feeb-4808-bf73-aaccdcd64272 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:04:34.100337 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1191/2349] 10.222.0.46 () {58 vars in 1121 bytes} [Thu Jun 23 16:04:34 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 16:04:34.105069 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ad5f8f-47ff-4122-baa6-ff4ea5337e7a admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:04:34.116026 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ad5f8f-47ff-4122-baa6-ff4ea5337e7a admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:04:34.137390 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e91046-bbc5-43da-8026-d22e878ba2b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:04:34.142995 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e91046-bbc5-43da-8026-d22e878ba2b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:04:34.143637 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e91046-bbc5-43da-8026-d22e878ba2b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:04:34.144141 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e91046-bbc5-43da-8026-d22e878ba2b1 admin admin] PATH_INFO: `/v3/projects/46a39c4592f044408de7c6414c77f7f9/users/fd6d74aa8bf244d0b37f34e718e4f132/roles/812d3c567c52469cb2b35709fbdc857c` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:04:34.146581 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e91046-bbc5-43da-8026-d22e878ba2b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=46a39c4592f044408de7c6414c77f7f9, user_id=fd6d74aa8bf244d0b37f34e718e4f132, role_id=812d3c567c52469cb2b35709fbdc857c)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:04:34.150291 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e91046-bbc5-43da-8026-d22e878ba2b1 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:04:34.161702 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1159/2350] 10.222.0.46 () {60 vars in 1482 bytes} [Thu Jun 23 16:04:34 2022] PUT /identity/v3/projects/46a39c4592f044408de7c6414c77f7f9/users/fd6d74aa8bf244d0b37f34e718e4f132/roles/812d3c567c52469cb2b35709fbdc857c => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 16:04:38.850544 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d3ccc9-73f1-4364-a4f6-f08460fde304 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:04:38.851656 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d3ccc9-73f1-4364-a4f6-f08460fde304 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:04:38.851767 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d3ccc9-73f1-4364-a4f6-f08460fde304 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:04:38.882126 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-77d3ccc9-73f1-4364-a4f6-f08460fde304 None None] MFA Rules not processed for user `fd6d74aa8bf244d0b37f34e718e4f132`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:04:38.883615 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-77d3ccc9-73f1-4364-a4f6-f08460fde304 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:04:38.962291 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1192/2351] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:04:38 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 16:04:38.976825 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e91046-bbc5-43da-8026-d22e878ba2b1 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:04:38.978647 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-b9e91046-bbc5-43da-8026-d22e878ba2b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:04:39.074871 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:04:39.079312 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e91046-bbc5-43da-8026-d22e878ba2b1 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:04:39.086107 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20618c03-de82-44af-acbd-e09fa75d7c81 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '705115d19bab424d894bea8b1b48a7f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:04:39.086823 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20618c03-de82-44af-acbd-e09fa75d7c81 service nova] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:04:39.087121 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20618c03-de82-44af-acbd-e09fa75d7c81 service nova] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:04:39.087429 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20618c03-de82-44af-acbd-e09fa75d7c81 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:04:39.154235 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20618c03-de82-44af-acbd-e09fa75d7c81 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:04:39.155883 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20618c03-de82-44af-acbd-e09fa75d7c81 service nova] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:04:39.165719 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1160/2352] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jun 23 16:04:38 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 16:04:45.078765 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d3ccc9-73f1-4364-a4f6-f08460fde304 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:04:45.080286 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-77d3ccc9-73f1-4364-a4f6-f08460fde304 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:04:45.137896 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d3ccc9-73f1-4364-a4f6-f08460fde304 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:04:45.141515 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c29cf8-64c2-4b52-93ce-0ca289df5660 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51784064951a4619ae09b9167fb0fd0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:04:45.142169 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c29cf8-64c2-4b52-93ce-0ca289df5660 service neutron] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:04:45.142453 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c29cf8-64c2-4b52-93ce-0ca289df5660 service neutron] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:04:45.142762 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c29cf8-64c2-4b52-93ce-0ca289df5660 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:04:45.144547 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-f1c29cf8-64c2-4b52-93ce-0ca289df5660 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:04:45.164113 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:04:45.209589 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c29cf8-64c2-4b52-93ce-0ca289df5660 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:04:45.211001 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c29cf8-64c2-4b52-93ce-0ca289df5660 service neutron] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:04:45.221155 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1193/2353] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jun 23 16:04:45 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 16:04:49.160700 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=27 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:04:55.216754 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=20 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:05:07.073083 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a655a1-c558-4d00-a5aa-3060de04f85a None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:05:07.073638 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a655a1-c558-4d00-a5aa-3060de04f85a None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:05:07.073638 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a655a1-c558-4d00-a5aa-3060de04f85a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:05:07.090983 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-b2a655a1-c558-4d00-a5aa-3060de04f85a None None] MFA Rules not processed for user `c953617a01f1401c966c16467138d628`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:05:07.092177 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-b2a655a1-c558-4d00-a5aa-3060de04f85a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:05:07.099038 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1161/2354] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:05:07 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 16:05:17.085718 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=3 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:05:19.874396 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998103fc-da19-400b-aed5-f333316b2c56 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:05:19.874941 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998103fc-da19-400b-aed5-f333316b2c56 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:05:19.875059 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998103fc-da19-400b-aed5-f333316b2c56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:05:19.893400 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-998103fc-da19-400b-aed5-f333316b2c56 None None] MFA Rules not processed for user `dac5e323b42f41859c85709a2b49623b`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:05:19.894836 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-998103fc-da19-400b-aed5-f333316b2c56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:05:19.951947 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1194/2355] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:05:19 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 16:05:22.625698 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b17357a-e7e5-45b4-8a07-6e239d2f21ae None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:05:22.626218 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b17357a-e7e5-45b4-8a07-6e239d2f21ae None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:05:22.626322 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b17357a-e7e5-45b4-8a07-6e239d2f21ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:05:22.645476 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-3b17357a-e7e5-45b4-8a07-6e239d2f21ae None None] MFA Rules not processed for user `be48addfaa154555b8d0cb7aac8b32ac`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:05:22.646780 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-3b17357a-e7e5-45b4-8a07-6e239d2f21ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:05:22.710526 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1162/2356] 10.222.0.46 () {58 vars in 1009 bytes} [Thu Jun 23 16:05:22 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 16:05:22.734336 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998103fc-da19-400b-aed5-f333316b2c56 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:05:22.736932 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-998103fc-da19-400b-aed5-f333316b2c56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:05:22.869723 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998103fc-da19-400b-aed5-f333316b2c56 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:05:22.876533 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41aa567-71cd-4356-8a88-da700eb26a8e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '242bf02d14724fe4837fcce8730377f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:05:22.877473 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41aa567-71cd-4356-8a88-da700eb26a8e service swift] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:05:22.877757 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41aa567-71cd-4356-8a88-da700eb26a8e service swift] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:05:22.878040 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41aa567-71cd-4356-8a88-da700eb26a8e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:05:22.891348 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41aa567-71cd-4356-8a88-da700eb26a8e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:05:22.893521 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41aa567-71cd-4356-8a88-da700eb26a8e service swift] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:05:22.903586 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1195/2357] 10.222.0.46 () {60 vars in 1384 bytes} [Thu Jun 23 16:05:22 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 23 16:05:32.175091 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b17357a-e7e5-45b4-8a07-6e239d2f21ae None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:05:32.185594 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b17357a-e7e5-45b4-8a07-6e239d2f21ae None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:05:32.189454 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fcfb16-68d6-4eea-8313-94bd8e5f8e35 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51784064951a4619ae09b9167fb0fd0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:05:32.190235 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fcfb16-68d6-4eea-8313-94bd8e5f8e35 service neutron] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:05:32.190667 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fcfb16-68d6-4eea-8313-94bd8e5f8e35 service neutron] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:05:32.191057 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fcfb16-68d6-4eea-8313-94bd8e5f8e35 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:05:32.201415 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fcfb16-68d6-4eea-8313-94bd8e5f8e35 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:05:32.202878 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fcfb16-68d6-4eea-8313-94bd8e5f8e35 service neutron] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:05:32.213410 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1163/2358] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jun 23 16:05:32 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 16:05:32.898855 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=62 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:05:41.283774 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031a4c9b-45fb-4eb9-b6ce-1ec639b5a523 None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:05:41.284823 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031a4c9b-45fb-4eb9-b6ce-1ec639b5a523 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:05:41.285104 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031a4c9b-45fb-4eb9-b6ce-1ec639b5a523 None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:05:41.286744 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1196/2359] 10.222.0.46 () {54 vars in 921 bytes} [Thu Jun 23 16:05:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 16:05:41.293376 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba2421a-0921-45af-a5ac-673f899233ec None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:05:41.293865 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba2421a-0921-45af-a5ac-673f899233ec None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:05:41.294066 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba2421a-0921-45af-a5ac-673f899233ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:05:41.332907 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-eba2421a-0921-45af-a5ac-673f899233ec None None] MFA Rules not processed for user `705115d19bab424d894bea8b1b48a7f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:05:41.334182 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-eba2421a-0921-45af-a5ac-673f899233ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:05:41.344139 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1164/2360] 10.222.0.46 () {58 vars in 1018 bytes} [Thu Jun 23 16:05:41 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 16:05:41.359283 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031a4c9b-45fb-4eb9-b6ce-1ec639b5a523 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:05:41.361171 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-031a4c9b-45fb-4eb9-b6ce-1ec639b5a523 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:05:41.444354 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031a4c9b-45fb-4eb9-b6ce-1ec639b5a523 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:05:41.447794 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3445aceb-a307-456f-b214-1d44a45a338d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83970271286e451e9008bbf07cd3fa86', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:05:41.448435 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3445aceb-a307-456f-b214-1d44a45a338d service glance] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:05:41.448712 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3445aceb-a307-456f-b214-1d44a45a338d service glance] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:05:41.448985 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3445aceb-a307-456f-b214-1d44a45a338d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:05:41.458961 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3445aceb-a307-456f-b214-1d44a45a338d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:05:41.460411 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3445aceb-a307-456f-b214-1d44a45a338d service glance] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:05:41.471371 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1197/2361] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jun 23 16:05:41 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 16:05:41.538650 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1eda852-de0d-4717-b4fe-7619d99fabc2 None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:05:41.538963 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1eda852-de0d-4717-b4fe-7619d99fabc2 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:05:41.539270 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1eda852-de0d-4717-b4fe-7619d99fabc2 None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:05:41.540666 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1165/2362] 10.222.0.46 () {54 vars in 921 bytes} [Thu Jun 23 16:05:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 16:05:41.546193 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176c9e45-2451-4d8b-a535-d645b8ed0b84 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:05:41.546478 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176c9e45-2451-4d8b-a535-d645b8ed0b84 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:05:41.546779 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176c9e45-2451-4d8b-a535-d645b8ed0b84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:05:41.564984 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-176c9e45-2451-4d8b-a535-d645b8ed0b84 None None] MFA Rules not processed for user `51784064951a4619ae09b9167fb0fd0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:05:41.566190 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-176c9e45-2451-4d8b-a535-d645b8ed0b84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:05:41.574488 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1198/2363] 10.222.0.46 () {58 vars in 1018 bytes} [Thu Jun 23 16:05:41 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 16:05:41.586315 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1eda852-de0d-4717-b4fe-7619d99fabc2 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:05:41.597875 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1eda852-de0d-4717-b4fe-7619d99fabc2 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:05:41.601481 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5a3ecd-45b5-4f12-a4da-9b397fe139f3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51784064951a4619ae09b9167fb0fd0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:05:41.602117 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5a3ecd-45b5-4f12-a4da-9b397fe139f3 service neutron] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:05:41.602400 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5a3ecd-45b5-4f12-a4da-9b397fe139f3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:05:41.602676 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5a3ecd-45b5-4f12-a4da-9b397fe139f3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:05:41.610364 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5a3ecd-45b5-4f12-a4da-9b397fe139f3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:05:41.611844 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5a3ecd-45b5-4f12-a4da-9b397fe139f3 service neutron] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:05:41.619666 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1166/2364] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jun 23 16:05:41 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 16:05:41.699930 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8e2b43-51fe-42c8-ae13-130fec8b16ab None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:05:41.700158 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8e2b43-51fe-42c8-ae13-130fec8b16ab None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:05:41.700440 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8e2b43-51fe-42c8-ae13-130fec8b16ab None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:05:41.701817 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1199/2365] 10.222.0.46 () {54 vars in 921 bytes} [Thu Jun 23 16:05:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 16:05:41.706769 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e88625c-221e-47c6-ad74-a271bf3edeca None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:05:41.707061 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e88625c-221e-47c6-ad74-a271bf3edeca None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:05:41.707338 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e88625c-221e-47c6-ad74-a271bf3edeca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:05:41.726508 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-9e88625c-221e-47c6-ad74-a271bf3edeca None None] MFA Rules not processed for user `705115d19bab424d894bea8b1b48a7f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:05:41.727746 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-9e88625c-221e-47c6-ad74-a271bf3edeca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:05:41.737587 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1167/2366] 10.222.0.46 () {58 vars in 1018 bytes} [Thu Jun 23 16:05:41 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 16:05:41.748217 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8e2b43-51fe-42c8-ae13-130fec8b16ab None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:05:41.749934 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-fb8e2b43-51fe-42c8-ae13-130fec8b16ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:05:41.763509 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8e2b43-51fe-42c8-ae13-130fec8b16ab None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:05:41.767042 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386d4327-19bc-4c1f-9d71-cbbf83647337 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51784064951a4619ae09b9167fb0fd0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:05:41.767674 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386d4327-19bc-4c1f-9d71-cbbf83647337 service neutron] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:05:41.767955 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386d4327-19bc-4c1f-9d71-cbbf83647337 service neutron] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:05:41.768227 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386d4327-19bc-4c1f-9d71-cbbf83647337 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:05:41.779357 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386d4327-19bc-4c1f-9d71-cbbf83647337 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:05:41.780780 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386d4327-19bc-4c1f-9d71-cbbf83647337 service neutron] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:05:41.791424 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1200/2367] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jun 23 16:05:41 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 16:05:44.336922 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3078d0cd-1829-4186-be1d-e8e4e68db76e None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:05:44.337480 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3078d0cd-1829-4186-be1d-e8e4e68db76e None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:05:44.337480 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3078d0cd-1829-4186-be1d-e8e4e68db76e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:05:44.357559 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-3078d0cd-1829-4186-be1d-e8e4e68db76e None None] MFA Rules not processed for user `be48addfaa154555b8d0cb7aac8b32ac`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:05:44.358975 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-3078d0cd-1829-4186-be1d-e8e4e68db76e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:05:44.366495 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1168/2368] 10.222.0.46 () {58 vars in 1009 bytes} [Thu Jun 23 16:05:44 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 16:05:44.378953 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386d4327-19bc-4c1f-9d71-cbbf83647337 service neutron] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:05:44.392518 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386d4327-19bc-4c1f-9d71-cbbf83647337 service neutron] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:05:44.398577 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2288bdf8-058f-4c68-bed5-02f9356ed40c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '242bf02d14724fe4837fcce8730377f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:05:44.399558 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2288bdf8-058f-4c68-bed5-02f9356ed40c service swift] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:05:44.400079 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2288bdf8-058f-4c68-bed5-02f9356ed40c service swift] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:05:44.400377 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2288bdf8-058f-4c68-bed5-02f9356ed40c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:05:44.412648 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2288bdf8-058f-4c68-bed5-02f9356ed40c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:05:44.414277 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2288bdf8-058f-4c68-bed5-02f9356ed40c service swift] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:05:44.424299 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1201/2369] 10.222.0.46 () {60 vars in 1384 bytes} [Thu Jun 23 16:05:44 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 23 16:05:48.479073 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3078d0cd-1829-4186-be1d-e8e4e68db76e None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:05:48.484312 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=53 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:05:48.490792 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3078d0cd-1829-4186-be1d-e8e4e68db76e None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:05:48.494383 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793b624d-f279-4a7e-844a-42f83d1910a3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51784064951a4619ae09b9167fb0fd0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:05:48.495081 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793b624d-f279-4a7e-844a-42f83d1910a3 service neutron] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:05:48.495365 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793b624d-f279-4a7e-844a-42f83d1910a3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:05:48.495640 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793b624d-f279-4a7e-844a-42f83d1910a3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:05:48.497411 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-793b624d-f279-4a7e-844a-42f83d1910a3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:05:48.512042 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793b624d-f279-4a7e-844a-42f83d1910a3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:05:48.513493 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793b624d-f279-4a7e-844a-42f83d1910a3 service neutron] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:05:48.525980 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1169/2370] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jun 23 16:05:48 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 16:05:49.703037 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2288bdf8-058f-4c68-bed5-02f9356ed40c service swift] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:05:49.713133 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2288bdf8-058f-4c68-bed5-02f9356ed40c service swift] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:05:49.717481 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebee2464-e338-45a5-be3c-0b47b38df295 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51784064951a4619ae09b9167fb0fd0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:05:49.718123 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebee2464-e338-45a5-be3c-0b47b38df295 service neutron] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:05:49.718404 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebee2464-e338-45a5-be3c-0b47b38df295 service neutron] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:05:49.718675 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebee2464-e338-45a5-be3c-0b47b38df295 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:05:49.720489 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-ebee2464-e338-45a5-be3c-0b47b38df295 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:05:49.734735 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebee2464-e338-45a5-be3c-0b47b38df295 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:05:49.736199 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebee2464-e338-45a5-be3c-0b47b38df295 service neutron] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:05:49.748182 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1202/2371] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jun 23 16:05:49 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 16:05:58.521035 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=5 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:05:59.743197 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=56 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:06:02.749225 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793b624d-f279-4a7e-844a-42f83d1910a3 service neutron] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:06:02.760849 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793b624d-f279-4a7e-844a-42f83d1910a3 service neutron] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:06:02.766336 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f9e961-5e5f-40dc-b962-cb095dfb8a50 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '705115d19bab424d894bea8b1b48a7f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:06:02.767086 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f9e961-5e5f-40dc-b962-cb095dfb8a50 service nova] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:06:02.767371 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f9e961-5e5f-40dc-b962-cb095dfb8a50 service nova] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:06:02.767648 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f9e961-5e5f-40dc-b962-cb095dfb8a50 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:06:02.769403 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-90f9e961-5e5f-40dc-b962-cb095dfb8a50 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:06:02.833852 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f9e961-5e5f-40dc-b962-cb095dfb8a50 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:06:02.835542 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f9e961-5e5f-40dc-b962-cb095dfb8a50 service nova] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:06:02.846348 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1170/2372] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jun 23 16:06:02 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 16:06:12.841622 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=28 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:06:13.517558 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebee2464-e338-45a5-be3c-0b47b38df295 service neutron] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:06:13.527593 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebee2464-e338-45a5-be3c-0b47b38df295 service neutron] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:06:13.531199 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880dfff9-c078-4af5-87fc-7584c7361299 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51784064951a4619ae09b9167fb0fd0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:06:13.531832 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880dfff9-c078-4af5-87fc-7584c7361299 service neutron] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:06:13.532134 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880dfff9-c078-4af5-87fc-7584c7361299 service neutron] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:06:13.532400 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880dfff9-c078-4af5-87fc-7584c7361299 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:06:13.534270 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-880dfff9-c078-4af5-87fc-7584c7361299 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:06:13.544343 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880dfff9-c078-4af5-87fc-7584c7361299 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:06:13.545777 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880dfff9-c078-4af5-87fc-7584c7361299 service neutron] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:06:13.554301 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1203/2373] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jun 23 16:06:13 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 16:06:17.565422 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aab3652-a1b8-4444-a9c0-14ed87732930 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:06:17.565884 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aab3652-a1b8-4444-a9c0-14ed87732930 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:06:17.565983 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aab3652-a1b8-4444-a9c0-14ed87732930 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:06:17.583932 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-1aab3652-a1b8-4444-a9c0-14ed87732930 None None] MFA Rules not processed for user `be48addfaa154555b8d0cb7aac8b32ac`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:06:17.585135 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-1aab3652-a1b8-4444-a9c0-14ed87732930 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:06:17.592264 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1171/2374] 10.222.0.46 () {58 vars in 967 bytes} [Thu Jun 23 16:06:17 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 16:06:17.604495 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880dfff9-c078-4af5-87fc-7584c7361299 service neutron] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:06:17.616619 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880dfff9-c078-4af5-87fc-7584c7361299 service neutron] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:06:17.622492 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6717b0b7-2f3a-4b0a-8859-adce4d85322f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '242bf02d14724fe4837fcce8730377f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:06:17.623178 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6717b0b7-2f3a-4b0a-8859-adce4d85322f service swift] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:06:17.623466 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6717b0b7-2f3a-4b0a-8859-adce4d85322f service swift] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:06:17.623750 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6717b0b7-2f3a-4b0a-8859-adce4d85322f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:06:17.632260 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6717b0b7-2f3a-4b0a-8859-adce4d85322f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:06:17.633882 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6717b0b7-2f3a-4b0a-8859-adce4d85322f service swift] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:06:17.642806 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1204/2375] 10.222.0.46 () {60 vars in 1384 bytes} [Thu Jun 23 16:06:17 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 23 16:06:22.064968 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aab3652-a1b8-4444-a9c0-14ed87732930 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:06:22.066862 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-1aab3652-a1b8-4444-a9c0-14ed87732930 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:06:22.083211 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aab3652-a1b8-4444-a9c0-14ed87732930 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:06:22.088787 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f0e564-fd64-45ad-b453-d322f87e398e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '705115d19bab424d894bea8b1b48a7f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:06:22.089459 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f0e564-fd64-45ad-b453-d322f87e398e service nova] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:06:22.089747 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f0e564-fd64-45ad-b453-d322f87e398e service nova] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:06:22.090028 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f0e564-fd64-45ad-b453-d322f87e398e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:06:22.091804 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-40f0e564-fd64-45ad-b453-d322f87e398e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:06:22.151102 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f0e564-fd64-45ad-b453-d322f87e398e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:06:22.152769 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f0e564-fd64-45ad-b453-d322f87e398e service nova] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:06:22.161995 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1172/2376] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jun 23 16:06:22 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 16:06:27.639334 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=12 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:06:32.157745 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=31 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:06:51.483544 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11417071-8642-45d4-9c0a-06364f086b40 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:06:51.484087 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11417071-8642-45d4-9c0a-06364f086b40 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:06:51.484087 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11417071-8642-45d4-9c0a-06364f086b40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:06:51.502848 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-11417071-8642-45d4-9c0a-06364f086b40 None None] MFA Rules not processed for user `228e5acf4578494ab1a28e4f530233f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:06:51.504081 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-11417071-8642-45d4-9c0a-06364f086b40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:06:51.511215 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1205/2377] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:06:51 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 16:06:58.338879 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f0e564-fd64-45ad-b453-d322f87e398e service nova] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:06:58.351662 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f0e564-fd64-45ad-b453-d322f87e398e service nova] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:06:58.358633 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370f4fc0-b791-43ed-8d36-8ba66c4cc0fa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '705115d19bab424d894bea8b1b48a7f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:06:58.359705 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370f4fc0-b791-43ed-8d36-8ba66c4cc0fa service nova] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:06:58.360310 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370f4fc0-b791-43ed-8d36-8ba66c4cc0fa service nova] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:06:58.360809 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370f4fc0-b791-43ed-8d36-8ba66c4cc0fa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:06:58.362928 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-370f4fc0-b791-43ed-8d36-8ba66c4cc0fa service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:06:58.381505 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370f4fc0-b791-43ed-8d36-8ba66c4cc0fa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:06:58.383506 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370f4fc0-b791-43ed-8d36-8ba66c4cc0fa service nova] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:06:58.396547 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1173/2378] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jun 23 16:06:58 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 16:07:01.497717 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=3 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:07:08.391631 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=6 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:07:19.484156 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11417071-8642-45d4-9c0a-06364f086b40 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:07:19.497438 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11417071-8642-45d4-9c0a-06364f086b40 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:07:19.503589 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8a7735-76ad-482a-b34c-506c1c442a9a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '705115d19bab424d894bea8b1b48a7f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:07:19.504236 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8a7735-76ad-482a-b34c-506c1c442a9a service nova] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:07:19.504521 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8a7735-76ad-482a-b34c-506c1c442a9a service nova] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:07:19.504800 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8a7735-76ad-482a-b34c-506c1c442a9a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:07:19.506584 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-1b8a7735-76ad-482a-b34c-506c1c442a9a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:07:19.516939 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8a7735-76ad-482a-b34c-506c1c442a9a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:07:19.518535 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8a7735-76ad-482a-b34c-506c1c442a9a service nova] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:07:19.527819 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1206/2379] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jun 23 16:07:19 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 16:07:22.621394 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370f4fc0-b791-43ed-8d36-8ba66c4cc0fa service nova] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:07:22.639434 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370f4fc0-b791-43ed-8d36-8ba66c4cc0fa service nova] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:07:22.652444 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49f4160-3b87-40bd-a201-cb97c459291d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '705115d19bab424d894bea8b1b48a7f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:07:22.654530 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49f4160-3b87-40bd-a201-cb97c459291d service nova] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:07:22.655821 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49f4160-3b87-40bd-a201-cb97c459291d service nova] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:07:22.656218 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49f4160-3b87-40bd-a201-cb97c459291d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:07:22.658749 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-c49f4160-3b87-40bd-a201-cb97c459291d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:07:22.676893 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49f4160-3b87-40bd-a201-cb97c459291d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:07:22.678826 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49f4160-3b87-40bd-a201-cb97c459291d service nova] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:07:22.692608 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1174/2380] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jun 23 16:07:22 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 72 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 16:07:29.523682 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=6 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:07:32.155405 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54804597-2af8-432b-83ae-ea361c3f52dd None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:07:32.156009 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54804597-2af8-432b-83ae-ea361c3f52dd None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:07:32.156009 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54804597-2af8-432b-83ae-ea361c3f52dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:07:32.174293 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-54804597-2af8-432b-83ae-ea361c3f52dd None None] MFA Rules not processed for user `dac5e323b42f41859c85709a2b49623b`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:07:32.175533 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-54804597-2af8-432b-83ae-ea361c3f52dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:07:32.182419 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1207/2381] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:07:32 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 16:07:32.480851 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49f4160-3b87-40bd-a201-cb97c459291d service nova] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:07:32.490837 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49f4160-3b87-40bd-a201-cb97c459291d service nova] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:07:32.494397 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c027c409-bf89-4fc2-869f-71d158de3bf4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51784064951a4619ae09b9167fb0fd0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:07:32.495146 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c027c409-bf89-4fc2-869f-71d158de3bf4 service neutron] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:07:32.495433 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c027c409-bf89-4fc2-869f-71d158de3bf4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:07:32.495711 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c027c409-bf89-4fc2-869f-71d158de3bf4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:07:32.497472 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-c027c409-bf89-4fc2-869f-71d158de3bf4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:07:32.507529 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c027c409-bf89-4fc2-869f-71d158de3bf4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:07:32.508939 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c027c409-bf89-4fc2-869f-71d158de3bf4 service neutron] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:07:32.516841 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1175/2382] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jun 23 16:07:32 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 16:07:32.828333 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54804597-2af8-432b-83ae-ea361c3f52dd None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:07:32.829844 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-54804597-2af8-432b-83ae-ea361c3f52dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:07:32.905715 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54804597-2af8-432b-83ae-ea361c3f52dd None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:07:32.910898 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc3ecf5-3d41-4d82-a8ea-d71b15adf129 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f66988b73644fe38b9849663ee70f25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:07:32.911592 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc3ecf5-3d41-4d82-a8ea-d71b15adf129 service placement] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:07:32.911878 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc3ecf5-3d41-4d82-a8ea-d71b15adf129 service placement] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:07:32.912156 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc3ecf5-3d41-4d82-a8ea-d71b15adf129 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:07:32.913955 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-2cc3ecf5-3d41-4d82-a8ea-d71b15adf129 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:07:32.926417 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc3ecf5-3d41-4d82-a8ea-d71b15adf129 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:07:32.928067 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc3ecf5-3d41-4d82-a8ea-d71b15adf129 service placement] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:07:32.938001 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1208/2383] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jun 23 16:07:32 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 16:07:41.438572 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c027c409-bf89-4fc2-869f-71d158de3bf4 service neutron] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:07:41.452289 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c027c409-bf89-4fc2-869f-71d158de3bf4 service neutron] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:07:41.458050 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62aa51a5-b66c-4f79-9050-b21216fadc30 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '705115d19bab424d894bea8b1b48a7f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:07:41.458733 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62aa51a5-b66c-4f79-9050-b21216fadc30 service nova] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:07:41.459024 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62aa51a5-b66c-4f79-9050-b21216fadc30 service nova] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:07:41.459305 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62aa51a5-b66c-4f79-9050-b21216fadc30 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:07:41.461252 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-62aa51a5-b66c-4f79-9050-b21216fadc30 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:07:41.473700 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62aa51a5-b66c-4f79-9050-b21216fadc30 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:07:41.475328 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62aa51a5-b66c-4f79-9050-b21216fadc30 service nova] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:07:41.486949 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1176/2384] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jun 23 16:07:41 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 16:07:42.933793 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=35 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:07:47.578056 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98432c63-96bc-4bd9-ac9c-c9c0f0589161 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:07:47.578056 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98432c63-96bc-4bd9-ac9c-c9c0f0589161 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:07:47.578056 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98432c63-96bc-4bd9-ac9c-c9c0f0589161 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:07:47.596777 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-98432c63-96bc-4bd9-ac9c-c9c0f0589161 None None] MFA Rules not processed for user `5dd4357556944c7089efb34ee58a8e04`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:07:47.597995 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-98432c63-96bc-4bd9-ac9c-c9c0f0589161 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:07:47.605164 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1209/2385] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:07:47 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 16:07:47.906452 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62aa51a5-b66c-4f79-9050-b21216fadc30 service nova] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:07:47.915966 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62aa51a5-b66c-4f79-9050-b21216fadc30 service nova] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:07:47.920715 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847763bf-4ab7-4bc4-882b-bb273108dafa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51784064951a4619ae09b9167fb0fd0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:07:47.921670 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847763bf-4ab7-4bc4-882b-bb273108dafa service neutron] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:07:47.922107 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847763bf-4ab7-4bc4-882b-bb273108dafa service neutron] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:07:47.922551 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847763bf-4ab7-4bc4-882b-bb273108dafa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:07:47.924571 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-847763bf-4ab7-4bc4-882b-bb273108dafa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:07:47.935543 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847763bf-4ab7-4bc4-882b-bb273108dafa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:07:47.937095 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847763bf-4ab7-4bc4-882b-bb273108dafa service neutron] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:07:47.945585 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1177/2386] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jun 23 16:07:47 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 16:07:53.053442 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f71afda-f357-4d58-a7b4-29f1230689dd None None] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:07:53.054035 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f71afda-f357-4d58-a7b4-29f1230689dd None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:07:53.054035 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f71afda-f357-4d58-a7b4-29f1230689dd None None] PATH_INFO: `/` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:07:53.055637 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1210/2387] 10.222.0.46 () {54 vars in 921 bytes} [Thu Jun 23 16:07:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 16:07:53.062996 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f9e34f-ea8a-4e44-9554-ce3a7e389fb9 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:07:53.063545 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f9e34f-ea8a-4e44-9554-ce3a7e389fb9 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:07:53.064706 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f9e34f-ea8a-4e44-9554-ce3a7e389fb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:07:53.071174 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=25 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:07:53.085454 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-c9f9e34f-ea8a-4e44-9554-ce3a7e389fb9 None None] MFA Rules not processed for user `51784064951a4619ae09b9167fb0fd0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:07:53.086703 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-c9f9e34f-ea8a-4e44-9554-ce3a7e389fb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:07:53.093450 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1178/2388] 10.222.0.46 () {58 vars in 1018 bytes} [Thu Jun 23 16:07:53 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 16:07:53.109606 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f71afda-f357-4d58-a7b4-29f1230689dd None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:07:53.120662 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f71afda-f357-4d58-a7b4-29f1230689dd None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:07:53.124858 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86ce9c3-5579-4f67-9b85-1bc8df03c624 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51784064951a4619ae09b9167fb0fd0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:07:53.125531 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86ce9c3-5579-4f67-9b85-1bc8df03c624 service neutron] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:07:53.125814 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86ce9c3-5579-4f67-9b85-1bc8df03c624 service neutron] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:07:53.126093 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86ce9c3-5579-4f67-9b85-1bc8df03c624 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:07:53.134185 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86ce9c3-5579-4f67-9b85-1bc8df03c624 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:07:53.135669 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86ce9c3-5579-4f67-9b85-1bc8df03c624 service neutron] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:07:53.144422 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1211/2389] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jun 23 16:07:53 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 16:07:53.247432 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ea363c-2cab-4e8d-9299-b1bc7d7e76c0 None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:07:53.248301 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ea363c-2cab-4e8d-9299-b1bc7d7e76c0 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:07:53.248579 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ea363c-2cab-4e8d-9299-b1bc7d7e76c0 None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:07:53.249942 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1179/2390] 10.222.0.46 () {54 vars in 921 bytes} [Thu Jun 23 16:07:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 16:07:53.255391 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315affde-ec06-46a8-bba0-8c29c6d9e4f3 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:07:53.255682 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315affde-ec06-46a8-bba0-8c29c6d9e4f3 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:07:53.255990 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315affde-ec06-46a8-bba0-8c29c6d9e4f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:07:53.274392 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-315affde-ec06-46a8-bba0-8c29c6d9e4f3 None None] MFA Rules not processed for user `705115d19bab424d894bea8b1b48a7f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:07:53.275593 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-315affde-ec06-46a8-bba0-8c29c6d9e4f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:07:53.284972 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1212/2391] 10.222.0.46 () {58 vars in 1018 bytes} [Thu Jun 23 16:07:53 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 16:07:53.296226 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ea363c-2cab-4e8d-9299-b1bc7d7e76c0 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:07:53.306997 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ea363c-2cab-4e8d-9299-b1bc7d7e76c0 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:07:53.311082 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9452ac7f-55ab-455d-b9a5-992157ed2c70 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51784064951a4619ae09b9167fb0fd0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:07:53.311727 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9452ac7f-55ab-455d-b9a5-992157ed2c70 service neutron] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:07:53.312011 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9452ac7f-55ab-455d-b9a5-992157ed2c70 service neutron] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:07:53.312289 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9452ac7f-55ab-455d-b9a5-992157ed2c70 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:07:53.323519 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9452ac7f-55ab-455d-b9a5-992157ed2c70 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:07:53.325269 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9452ac7f-55ab-455d-b9a5-992157ed2c70 service neutron] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:07:53.336346 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1180/2392] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jun 23 16:07:53 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 16:08:03.268941 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=12 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:08:03.331638 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=8 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:08:14.286730 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315affde-ec06-46a8-bba0-8c29c6d9e4f3 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:08:14.298338 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315affde-ec06-46a8-bba0-8c29c6d9e4f3 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:08:14.304505 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876dda1d-9f61-44de-ab59-268941662c06 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '705115d19bab424d894bea8b1b48a7f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:08:14.305154 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876dda1d-9f61-44de-ab59-268941662c06 service nova] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:08:14.305436 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876dda1d-9f61-44de-ab59-268941662c06 service nova] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:08:14.305711 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876dda1d-9f61-44de-ab59-268941662c06 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:08:14.307544 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-876dda1d-9f61-44de-ab59-268941662c06 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:08:14.369191 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876dda1d-9f61-44de-ab59-268941662c06 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:08:14.370957 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876dda1d-9f61-44de-ab59-268941662c06 service nova] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:08:14.381207 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1213/2393] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jun 23 16:08:14 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 16:08:14.545032 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9452ac7f-55ab-455d-b9a5-992157ed2c70 service neutron] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:08:14.557866 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9452ac7f-55ab-455d-b9a5-992157ed2c70 service neutron] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:08:14.565044 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57b0742-254c-4cc6-8fc6-54fae4ff9bff service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '705115d19bab424d894bea8b1b48a7f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:08:14.565842 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57b0742-254c-4cc6-8fc6-54fae4ff9bff service nova] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:08:14.566240 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57b0742-254c-4cc6-8fc6-54fae4ff9bff service nova] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:08:14.566645 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57b0742-254c-4cc6-8fc6-54fae4ff9bff service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:08:14.568791 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-f57b0742-254c-4cc6-8fc6-54fae4ff9bff service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:08:14.633632 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57b0742-254c-4cc6-8fc6-54fae4ff9bff service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:08:14.635294 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57b0742-254c-4cc6-8fc6-54fae4ff9bff service nova] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:08:14.645149 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1181/2394] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jun 23 16:08:14 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 16:08:20.332064 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a173e5a-01a0-427c-b9b2-0f823888d40b None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:08:20.332610 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a173e5a-01a0-427c-b9b2-0f823888d40b None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:08:20.332610 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a173e5a-01a0-427c-b9b2-0f823888d40b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:08:20.350099 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-3a173e5a-01a0-427c-b9b2-0f823888d40b None None] MFA Rules not processed for user `c953617a01f1401c966c16467138d628`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:08:20.351348 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-3a173e5a-01a0-427c-b9b2-0f823888d40b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:08:20.359656 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1214/2395] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:08:20 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 16:08:24.640844 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=28 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:08:24.873617 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57b0742-254c-4cc6-8fc6-54fae4ff9bff service nova] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:08:24.884033 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57b0742-254c-4cc6-8fc6-54fae4ff9bff service nova] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:08:24.888419 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44d8aea-388d-4e0f-b290-4776a83d86ac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51784064951a4619ae09b9167fb0fd0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:08:24.889239 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44d8aea-388d-4e0f-b290-4776a83d86ac service neutron] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:08:24.889637 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44d8aea-388d-4e0f-b290-4776a83d86ac service neutron] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:08:24.889961 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44d8aea-388d-4e0f-b290-4776a83d86ac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:08:24.892693 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-a44d8aea-388d-4e0f-b290-4776a83d86ac service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:08:24.970025 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44d8aea-388d-4e0f-b290-4776a83d86ac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:08:24.971442 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44d8aea-388d-4e0f-b290-4776a83d86ac service neutron] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:08:24.982206 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1182/2396] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jun 23 16:08:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 16:08:28.362064 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a173e5a-01a0-427c-b9b2-0f823888d40b None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:08:28.364810 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-3a173e5a-01a0-427c-b9b2-0f823888d40b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:08:28.389058 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a173e5a-01a0-427c-b9b2-0f823888d40b None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:08:28.394163 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abad2320-e92b-4000-a5e7-27316ab8df5f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f66988b73644fe38b9849663ee70f25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:08:28.394872 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abad2320-e92b-4000-a5e7-27316ab8df5f service placement] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:08:28.395322 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abad2320-e92b-4000-a5e7-27316ab8df5f service placement] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:08:28.395427 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abad2320-e92b-4000-a5e7-27316ab8df5f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:08:28.397240 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-abad2320-e92b-4000-a5e7-27316ab8df5f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:08:28.410961 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abad2320-e92b-4000-a5e7-27316ab8df5f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:08:28.412816 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abad2320-e92b-4000-a5e7-27316ab8df5f service placement] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:08:28.433210 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1215/2397] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jun 23 16:08:28 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 72 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 16:08:28.473586 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44d8aea-388d-4e0f-b290-4776a83d86ac service neutron] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:08:28.475229 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-a44d8aea-388d-4e0f-b290-4776a83d86ac service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:08:28.488556 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44d8aea-388d-4e0f-b290-4776a83d86ac service neutron] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:08:28.493156 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dd4392-1d93-4d46-b692-fcf6eb61e0f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:08:28.494030 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dd4392-1d93-4d46-b692-fcf6eb61e0f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:08:28.494310 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dd4392-1d93-4d46-b692-fcf6eb61e0f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:08:28.494588 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dd4392-1d93-4d46-b692-fcf6eb61e0f0 admin admin] PATH_INFO: `/v3/users/228e5acf4578494ab1a28e4f530233f2` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:08:28.495860 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8dd4392-1d93-4d46-b692-fcf6eb61e0f0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=228e5acf4578494ab1a28e4f530233f2)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:08:28.497765 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8dd4392-1d93-4d46-b692-fcf6eb61e0f0 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:08:28.548257 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-a8dd4392-1d93-4d46-b692-fcf6eb61e0f0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '228e5acf4578494ab1a28e4f530233f2'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:08:28.548915 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-a8dd4392-1d93-4d46-b692-fcf6eb61e0f0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '228e5acf4578494ab1a28e4f530233f2'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:08:28.555823 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-a8dd4392-1d93-4d46-b692-fcf6eb61e0f0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '228e5acf4578494ab1a28e4f530233f2'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:08:28.563865 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-a8dd4392-1d93-4d46-b692-fcf6eb61e0f0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '228e5acf4578494ab1a28e4f530233f2'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:08:28.564476 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-a8dd4392-1d93-4d46-b692-fcf6eb61e0f0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '228e5acf4578494ab1a28e4f530233f2'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:08:28.583155 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1183/2398] 10.222.0.46 () {58 vars in 1223 bytes} [Thu Jun 23 16:08:28 2022] DELETE /identity/v3/users/228e5acf4578494ab1a28e4f530233f2 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 16:08:28.763097 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abad2320-e92b-4000-a5e7-27316ab8df5f service placement] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:08:28.775857 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abad2320-e92b-4000-a5e7-27316ab8df5f service placement] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:08:28.837673 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9369dd8f-f57f-43ca-83ba-bbcce7cc96d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:08:28.839042 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9369dd8f-f57f-43ca-83ba-bbcce7cc96d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:08:28.839401 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9369dd8f-f57f-43ca-83ba-bbcce7cc96d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:08:28.839887 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9369dd8f-f57f-43ca-83ba-bbcce7cc96d6 admin admin] PATH_INFO: `/v3/projects/f9eeca1030964294b8d813812abdcc5b` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:08:28.841475 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9369dd8f-f57f-43ca-83ba-bbcce7cc96d6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f9eeca1030964294b8d813812abdcc5b)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:08:28.843535 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9369dd8f-f57f-43ca-83ba-bbcce7cc96d6 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:08:28.893696 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-9369dd8f-f57f-43ca-83ba-bbcce7cc96d6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f9eeca1030964294b8d813812abdcc5b', 'request_id': 'req-9369dd8f-f57f-43ca-83ba-bbcce7cc96d6'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:08:28.899339 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-9369dd8f-f57f-43ca-83ba-bbcce7cc96d6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f9eeca1030964294b8d813812abdcc5b', 'request_id': 'req-9369dd8f-f57f-43ca-83ba-bbcce7cc96d6'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:08:28.905540 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-9369dd8f-f57f-43ca-83ba-bbcce7cc96d6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f9eeca1030964294b8d813812abdcc5b', 'request_id': 'req-9369dd8f-f57f-43ca-83ba-bbcce7cc96d6'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:08:28.917872 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-9369dd8f-f57f-43ca-83ba-bbcce7cc96d6 admin admin] The token cache is being invalidate because project f9eeca1030964294b8d813812abdcc5b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80153) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 16:08:28.918161 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-9369dd8f-f57f-43ca-83ba-bbcce7cc96d6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:08:28.918831 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-9369dd8f-f57f-43ca-83ba-bbcce7cc96d6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:08:28.920387 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1216/2399] 10.222.0.46 () {58 vars in 1232 bytes} [Thu Jun 23 16:08:28 2022] DELETE /identity/v3/projects/f9eeca1030964294b8d813812abdcc5b => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 16:08:28.925341 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee5c165-67b8-4f43-89b7-fddac5ccb4db None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:08:28.925629 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee5c165-67b8-4f43-89b7-fddac5ccb4db None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:08:28.925906 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee5c165-67b8-4f43-89b7-fddac5ccb4db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:08:28.943750 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-2ee5c165-67b8-4f43-89b7-fddac5ccb4db None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:08:28.945007 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-2ee5c165-67b8-4f43-89b7-fddac5ccb4db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:08:29.005118 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1184/2400] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:08:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 16:08:29.013302 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c19876-e5c9-4bda-bcd5-d78385e9f466 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:08:29.013594 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c19876-e5c9-4bda-bcd5-d78385e9f466 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:08:29.013887 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c19876-e5c9-4bda-bcd5-d78385e9f466 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:08:29.033883 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-26c19876-e5c9-4bda-bcd5-d78385e9f466 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:08:29.035132 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-26c19876-e5c9-4bda-bcd5-d78385e9f466 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:08:29.043588 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1217/2401] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:08:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 16:08:29.048063 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee5c165-67b8-4f43-89b7-fddac5ccb4db None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:08:29.059533 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee5c165-67b8-4f43-89b7-fddac5ccb4db None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:08:29.065203 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a22b3ea-9e84-493f-8523-384c89387800 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:08:29.066749 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a22b3ea-9e84-493f-8523-384c89387800 admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:08:29.067244 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a22b3ea-9e84-493f-8523-384c89387800 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:08:29.067750 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a22b3ea-9e84-493f-8523-384c89387800 admin admin] PATH_INFO: `/v3/domains` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:08:29.068679 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a22b3ea-9e84-493f-8523-384c89387800 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 16:08:29.069175 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a22b3ea-9e84-493f-8523-384c89387800 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:08:29.071336 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a22b3ea-9e84-493f-8523-384c89387800 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:08:29.073263 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:08:29.087510 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1185/2402] 10.222.0.46 () {58 vars in 1152 bytes} [Thu Jun 23 16:08:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 16:08:29.092449 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c19876-e5c9-4bda-bcd5-d78385e9f466 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:08:29.097485 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:08:29.104421 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c19876-e5c9-4bda-bcd5-d78385e9f466 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:08:29.110366 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c03975-86c0-4868-b29c-b9b725111a7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:08:29.111091 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c03975-86c0-4868-b29c-b9b725111a7f admin admin] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:08:29.111373 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c03975-86c0-4868-b29c-b9b725111a7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:08:29.111645 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c03975-86c0-4868-b29c-b9b725111a7f admin admin] PATH_INFO: `/v3/projects` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:08:29.112339 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c03975-86c0-4868-b29c-b9b725111a7f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:08:29.114249 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c03975-86c0-4868-b29c-b9b725111a7f admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:08:29.156896 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1218/2403] 10.222.0.46 () {60 vars in 1152 bytes} [Thu Jun 23 16:08:29 2022] POST /identity/v3/projects => generated 373 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 16:08:29.160964 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a22b3ea-9e84-493f-8523-384c89387800 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:08:29.172667 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a22b3ea-9e84-493f-8523-384c89387800 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:08:29.232852 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0355d662-1082-4273-b686-adcb273e72cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:08:29.233534 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0355d662-1082-4273-b686-adcb273e72cc admin admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:08:29.233857 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0355d662-1082-4273-b686-adcb273e72cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:08:29.234136 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0355d662-1082-4273-b686-adcb273e72cc admin admin] PATH_INFO: `/v3/users` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:08:29.234864 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0355d662-1082-4273-b686-adcb273e72cc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:08:29.236833 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0355d662-1082-4273-b686-adcb273e72cc admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:08:29.257414 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1186/2404] 10.222.0.46 () {60 vars in 1143 bytes} [Thu Jun 23 16:08:29 2022] POST /identity/v3/users => generated 287 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 16:08:29.261547 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c03975-86c0-4868-b29c-b9b725111a7f admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:08:29.271827 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c03975-86c0-4868-b29c-b9b725111a7f admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:08:29.276704 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6358762c-17cb-4519-9e01-3807a2c5073a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:08:29.277361 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6358762c-17cb-4519-9e01-3807a2c5073a admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:08:29.277645 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6358762c-17cb-4519-9e01-3807a2c5073a admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:08:29.278060 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6358762c-17cb-4519-9e01-3807a2c5073a admin admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:08:29.278830 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6358762c-17cb-4519-9e01-3807a2c5073a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:08:29.280428 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6358762c-17cb-4519-9e01-3807a2c5073a admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:08:29.290303 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1219/2405] 10.222.0.46 () {58 vars in 1121 bytes} [Thu Jun 23 16:08:29 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 16:08:29.295800 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0355d662-1082-4273-b686-adcb273e72cc admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:08:29.305760 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0355d662-1082-4273-b686-adcb273e72cc admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:08:29.311113 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b36dc42-7190-4433-9517-953f87545aa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:08:29.311773 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b36dc42-7190-4433-9517-953f87545aa2 admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:08:29.312057 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b36dc42-7190-4433-9517-953f87545aa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:08:29.312332 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b36dc42-7190-4433-9517-953f87545aa2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:08:29.313057 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b36dc42-7190-4433-9517-953f87545aa2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:08:29.314648 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b36dc42-7190-4433-9517-953f87545aa2 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:08:29.324782 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1187/2406] 10.222.0.46 () {58 vars in 1121 bytes} [Thu Jun 23 16:08:29 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 16:08:29.329203 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6358762c-17cb-4519-9e01-3807a2c5073a admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:08:29.339155 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6358762c-17cb-4519-9e01-3807a2c5073a admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:08:29.343838 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a2b6ef-7cbd-4072-8acc-0efd2d1a4696 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:08:29.344732 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a2b6ef-7cbd-4072-8acc-0efd2d1a4696 admin admin] REQUEST_METHOD: `PUT` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:08:29.345011 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a2b6ef-7cbd-4072-8acc-0efd2d1a4696 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:08:29.345288 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a2b6ef-7cbd-4072-8acc-0efd2d1a4696 admin admin] PATH_INFO: `/v3/projects/73140d3e6cdc4295a8c694af753e59b5/users/abbac3c3fb8546abaf8b6ba207d79e7e/roles/e2a5bbf3200a4d4091d72bc7ff78c34d` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:08:29.360354 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a2b6ef-7cbd-4072-8acc-0efd2d1a4696 admin admin] RBAC: Authorizing `identity:create_grant(project_id=73140d3e6cdc4295a8c694af753e59b5, user_id=abbac3c3fb8546abaf8b6ba207d79e7e, role_id=e2a5bbf3200a4d4091d72bc7ff78c34d)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:08:29.363040 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a2b6ef-7cbd-4072-8acc-0efd2d1a4696 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:08:29.381306 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1220/2407] 10.222.0.46 () {60 vars in 1482 bytes} [Thu Jun 23 16:08:29 2022] PUT /identity/v3/projects/73140d3e6cdc4295a8c694af753e59b5/users/abbac3c3fb8546abaf8b6ba207d79e7e/roles/e2a5bbf3200a4d4091d72bc7ff78c34d => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 16:08:29.391824 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed870220-ece9-4e4a-be76-a423ebf92bbb None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:08:29.392125 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed870220-ece9-4e4a-be76-a423ebf92bbb None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:08:29.392414 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed870220-ece9-4e4a-be76-a423ebf92bbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:08:29.420167 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-ed870220-ece9-4e4a-be76-a423ebf92bbb None None] MFA Rules not processed for user `abbac3c3fb8546abaf8b6ba207d79e7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:08:29.421316 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-ed870220-ece9-4e4a-be76-a423ebf92bbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:08:29.488258 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1188/2408] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:08:29 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 16:08:29.615294 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a2b6ef-7cbd-4072-8acc-0efd2d1a4696 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:08:29.617094 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-59a2b6ef-7cbd-4072-8acc-0efd2d1a4696 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:08:29.693711 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a2b6ef-7cbd-4072-8acc-0efd2d1a4696 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:08:29.699183 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678c64c9-c4dc-49ea-8a20-824a1b66d30e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '705115d19bab424d894bea8b1b48a7f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:08:29.699881 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678c64c9-c4dc-49ea-8a20-824a1b66d30e service nova] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:08:29.700157 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678c64c9-c4dc-49ea-8a20-824a1b66d30e service nova] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:08:29.700429 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678c64c9-c4dc-49ea-8a20-824a1b66d30e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:08:29.708405 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678c64c9-c4dc-49ea-8a20-824a1b66d30e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:08:29.709990 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678c64c9-c4dc-49ea-8a20-824a1b66d30e service nova] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:08:29.720165 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1221/2409] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jun 23 16:08:29 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 16:08:29.976389 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed870220-ece9-4e4a-be76-a423ebf92bbb None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:08:30.039119 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed870220-ece9-4e4a-be76-a423ebf92bbb None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:08:30.044675 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32288e6-db55-4175-aae8-5a9cae618c3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:08:30.045567 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32288e6-db55-4175-aae8-5a9cae618c3f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:08:30.045858 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32288e6-db55-4175-aae8-5a9cae618c3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:08:30.046141 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32288e6-db55-4175-aae8-5a9cae618c3f admin admin] PATH_INFO: `/v3/users/abbac3c3fb8546abaf8b6ba207d79e7e` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:08:30.047385 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32288e6-db55-4175-aae8-5a9cae618c3f admin admin] RBAC: Authorizing `identity:delete_user(user_id=abbac3c3fb8546abaf8b6ba207d79e7e)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:08:30.049366 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32288e6-db55-4175-aae8-5a9cae618c3f admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:08:30.051922 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:08:30.101073 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-b32288e6-db55-4175-aae8-5a9cae618c3f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'abbac3c3fb8546abaf8b6ba207d79e7e'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:08:30.101740 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-b32288e6-db55-4175-aae8-5a9cae618c3f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'abbac3c3fb8546abaf8b6ba207d79e7e'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:08:30.109824 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-b32288e6-db55-4175-aae8-5a9cae618c3f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'abbac3c3fb8546abaf8b6ba207d79e7e'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:08:30.118119 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-b32288e6-db55-4175-aae8-5a9cae618c3f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'abbac3c3fb8546abaf8b6ba207d79e7e'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:08:30.118771 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-b32288e6-db55-4175-aae8-5a9cae618c3f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'abbac3c3fb8546abaf8b6ba207d79e7e'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:08:30.136992 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1189/2410] 10.222.0.46 () {58 vars in 1223 bytes} [Thu Jun 23 16:08:29 2022] DELETE /identity/v3/users/abbac3c3fb8546abaf8b6ba207d79e7e => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 16:08:30.142325 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765674bd-0c8f-4a91-a1cd-155160320732 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:08:30.142615 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765674bd-0c8f-4a91-a1cd-155160320732 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:08:30.142938 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765674bd-0c8f-4a91-a1cd-155160320732 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:08:30.161777 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-765674bd-0c8f-4a91-a1cd-155160320732 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:08:30.163033 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-765674bd-0c8f-4a91-a1cd-155160320732 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:08:30.225729 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1222/2411] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:08:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 16:08:30.235541 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32288e6-db55-4175-aae8-5a9cae618c3f admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:08:30.237204 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-b32288e6-db55-4175-aae8-5a9cae618c3f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:08:30.296270 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32288e6-db55-4175-aae8-5a9cae618c3f admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:08:30.300008 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57253005-0bf5-424a-b55d-1ac012320250 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51784064951a4619ae09b9167fb0fd0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:08:30.300668 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57253005-0bf5-424a-b55d-1ac012320250 service neutron] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:08:30.300948 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57253005-0bf5-424a-b55d-1ac012320250 service neutron] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:08:30.301224 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57253005-0bf5-424a-b55d-1ac012320250 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:08:30.311293 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57253005-0bf5-424a-b55d-1ac012320250 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:08:30.312831 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57253005-0bf5-424a-b55d-1ac012320250 service neutron] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:08:30.323072 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1190/2412] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jun 23 16:08:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 16:08:30.601562 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765674bd-0c8f-4a91-a1cd-155160320732 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:08:30.603198 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-765674bd-0c8f-4a91-a1cd-155160320732 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:08:30.616440 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765674bd-0c8f-4a91-a1cd-155160320732 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:08:30.621742 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986738ce-bf23-4312-8bee-b43f3a076fe3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:08:30.622621 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986738ce-bf23-4312-8bee-b43f3a076fe3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:08:30.622942 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986738ce-bf23-4312-8bee-b43f3a076fe3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:08:30.623205 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986738ce-bf23-4312-8bee-b43f3a076fe3 admin admin] PATH_INFO: `/v3/projects/73140d3e6cdc4295a8c694af753e59b5` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:08:30.624419 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986738ce-bf23-4312-8bee-b43f3a076fe3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=73140d3e6cdc4295a8c694af753e59b5)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:08:30.626377 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986738ce-bf23-4312-8bee-b43f3a076fe3 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:08:30.649723 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:08:30.682471 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-986738ce-bf23-4312-8bee-b43f3a076fe3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '73140d3e6cdc4295a8c694af753e59b5', 'request_id': 'req-986738ce-bf23-4312-8bee-b43f3a076fe3'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:08:30.688606 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-986738ce-bf23-4312-8bee-b43f3a076fe3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '73140d3e6cdc4295a8c694af753e59b5', 'request_id': 'req-986738ce-bf23-4312-8bee-b43f3a076fe3'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:08:30.694886 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-986738ce-bf23-4312-8bee-b43f3a076fe3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '73140d3e6cdc4295a8c694af753e59b5', 'request_id': 'req-986738ce-bf23-4312-8bee-b43f3a076fe3'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:08:30.709415 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-986738ce-bf23-4312-8bee-b43f3a076fe3 admin admin] The token cache is being invalidate because project 73140d3e6cdc4295a8c694af753e59b5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80153) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 16:08:30.709709 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-986738ce-bf23-4312-8bee-b43f3a076fe3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:08:30.710428 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-986738ce-bf23-4312-8bee-b43f3a076fe3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:08:30.712554 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1223/2413] 10.222.0.46 () {58 vars in 1232 bytes} [Thu Jun 23 16:08:30 2022] DELETE /identity/v3/projects/73140d3e6cdc4295a8c694af753e59b5 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 16:08:30.719849 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b8e65b-f832-409f-b13a-4540b582a97b None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:08:30.720375 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b8e65b-f832-409f-b13a-4540b582a97b None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:08:30.720488 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b8e65b-f832-409f-b13a-4540b582a97b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:08:30.738235 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-40b8e65b-f832-409f-b13a-4540b582a97b None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:08:30.739482 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-40b8e65b-f832-409f-b13a-4540b582a97b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:08:30.801877 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1191/2414] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:08:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 16:08:30.809588 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a262065d-0b63-4fa7-8366-2b0ee583b987 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:08:30.809876 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a262065d-0b63-4fa7-8366-2b0ee583b987 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:08:30.810157 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a262065d-0b63-4fa7-8366-2b0ee583b987 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:08:30.829516 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-a262065d-0b63-4fa7-8366-2b0ee583b987 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:08:30.830866 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-a262065d-0b63-4fa7-8366-2b0ee583b987 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:08:30.845093 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1224/2415] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:08:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 16:08:30.849538 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b8e65b-f832-409f-b13a-4540b582a97b None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:08:30.860635 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b8e65b-f832-409f-b13a-4540b582a97b None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:08:30.865361 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a0622e-b882-43ff-86a8-cc60c521e00d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:08:30.866031 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a0622e-b882-43ff-86a8-cc60c521e00d admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:08:30.866319 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a0622e-b882-43ff-86a8-cc60c521e00d admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:08:30.866598 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a0622e-b882-43ff-86a8-cc60c521e00d admin admin] PATH_INFO: `/v3/domains` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:08:30.867367 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a0622e-b882-43ff-86a8-cc60c521e00d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 16:08:30.867677 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a0622e-b882-43ff-86a8-cc60c521e00d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:08:30.869370 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a0622e-b882-43ff-86a8-cc60c521e00d admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:08:30.881373 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1192/2416] 10.222.0.46 () {58 vars in 1152 bytes} [Thu Jun 23 16:08:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 16:08:30.885997 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a262065d-0b63-4fa7-8366-2b0ee583b987 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:08:30.895311 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a262065d-0b63-4fa7-8366-2b0ee583b987 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:08:30.901818 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f26a16f-a7a5-44ef-8000-eeb4a4d0ec08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:08:30.902837 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f26a16f-a7a5-44ef-8000-eeb4a4d0ec08 admin admin] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:08:30.903734 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f26a16f-a7a5-44ef-8000-eeb4a4d0ec08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:08:30.904394 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f26a16f-a7a5-44ef-8000-eeb4a4d0ec08 admin admin] PATH_INFO: `/v3/projects` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:08:30.905460 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f26a16f-a7a5-44ef-8000-eeb4a4d0ec08 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:08:30.909240 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f26a16f-a7a5-44ef-8000-eeb4a4d0ec08 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:08:30.937903 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1225/2417] 10.222.0.46 () {60 vars in 1152 bytes} [Thu Jun 23 16:08:30 2022] POST /identity/v3/projects => generated 389 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 16:08:30.942265 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a0622e-b882-43ff-86a8-cc60c521e00d admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:08:30.952953 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a0622e-b882-43ff-86a8-cc60c521e00d admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:08:30.962306 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:08:31.016265 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bed37a-3e52-4925-bca9-3a8ad24a3fd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:08:31.016947 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bed37a-3e52-4925-bca9-3a8ad24a3fd5 admin admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:08:31.017232 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bed37a-3e52-4925-bca9-3a8ad24a3fd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:08:31.017511 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bed37a-3e52-4925-bca9-3a8ad24a3fd5 admin admin] PATH_INFO: `/v3/users` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:08:31.018293 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47bed37a-3e52-4925-bca9-3a8ad24a3fd5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:08:31.020332 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47bed37a-3e52-4925-bca9-3a8ad24a3fd5 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:08:31.048622 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1193/2418] 10.222.0.46 () {60 vars in 1143 bytes} [Thu Jun 23 16:08:30 2022] POST /identity/v3/users => generated 295 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 16:08:31.052842 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f26a16f-a7a5-44ef-8000-eeb4a4d0ec08 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:08:31.063444 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f26a16f-a7a5-44ef-8000-eeb4a4d0ec08 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:08:31.068256 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fc5f5f-10d7-4621-988f-1af064204e78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:08:31.069005 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fc5f5f-10d7-4621-988f-1af064204e78 admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:08:31.069360 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fc5f5f-10d7-4621-988f-1af064204e78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:08:31.069709 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fc5f5f-10d7-4621-988f-1af064204e78 admin admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:08:31.070513 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fc5f5f-10d7-4621-988f-1af064204e78 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:08:31.072265 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fc5f5f-10d7-4621-988f-1af064204e78 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:08:31.083834 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1226/2419] 10.222.0.46 () {58 vars in 1121 bytes} [Thu Jun 23 16:08:31 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 16:08:31.087729 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bed37a-3e52-4925-bca9-3a8ad24a3fd5 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:08:31.098730 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bed37a-3e52-4925-bca9-3a8ad24a3fd5 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:08:31.104429 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69ca8f0-0295-417f-992a-082550d7891d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:08:31.105115 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69ca8f0-0295-417f-992a-082550d7891d admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:08:31.105438 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69ca8f0-0295-417f-992a-082550d7891d admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:08:31.105714 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69ca8f0-0295-417f-992a-082550d7891d admin admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:08:31.106921 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69ca8f0-0295-417f-992a-082550d7891d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:08:31.108573 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69ca8f0-0295-417f-992a-082550d7891d admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:08:31.119680 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1194/2420] 10.222.0.46 () {58 vars in 1121 bytes} [Thu Jun 23 16:08:31 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 16:08:31.129363 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fc5f5f-10d7-4621-988f-1af064204e78 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:08:31.140430 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fc5f5f-10d7-4621-988f-1af064204e78 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:08:31.145202 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a589db-c0c2-4906-b402-bf9c41319e94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:08:31.146181 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a589db-c0c2-4906-b402-bf9c41319e94 admin admin] REQUEST_METHOD: `PUT` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:08:31.146465 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a589db-c0c2-4906-b402-bf9c41319e94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:08:31.146772 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a589db-c0c2-4906-b402-bf9c41319e94 admin admin] PATH_INFO: `/v3/projects/17226cb5d2ed43e1829e1399de364969/users/507b4fa5d05148abbeb66f3ea275a73e/roles/e2a5bbf3200a4d4091d72bc7ff78c34d` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:08:31.163536 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a589db-c0c2-4906-b402-bf9c41319e94 admin admin] RBAC: Authorizing `identity:create_grant(project_id=17226cb5d2ed43e1829e1399de364969, user_id=507b4fa5d05148abbeb66f3ea275a73e, role_id=e2a5bbf3200a4d4091d72bc7ff78c34d)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:08:31.166207 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a589db-c0c2-4906-b402-bf9c41319e94 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:08:31.179585 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1227/2421] 10.222.0.46 () {60 vars in 1482 bytes} [Thu Jun 23 16:08:31 2022] PUT /identity/v3/projects/17226cb5d2ed43e1829e1399de364969/users/507b4fa5d05148abbeb66f3ea275a73e/roles/e2a5bbf3200a4d4091d72bc7ff78c34d => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 16:08:31.190398 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255211ff-7caa-426a-9a17-f74771e2f923 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:08:31.190725 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255211ff-7caa-426a-9a17-f74771e2f923 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:08:31.191025 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255211ff-7caa-426a-9a17-f74771e2f923 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:08:31.218380 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-255211ff-7caa-426a-9a17-f74771e2f923 None None] MFA Rules not processed for user `507b4fa5d05148abbeb66f3ea275a73e`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:08:31.219650 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-255211ff-7caa-426a-9a17-f74771e2f923 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:08:31.287047 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1195/2422] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:08:31 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 16:08:31.291531 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a589db-c0c2-4906-b402-bf9c41319e94 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:08:31.354798 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a589db-c0c2-4906-b402-bf9c41319e94 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:08:31.359592 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca244d95-9ecd-4200-9c7d-cf3cc20b0aa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:08:31.360257 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca244d95-9ecd-4200-9c7d-cf3cc20b0aa7 admin admin] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:08:31.360542 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca244d95-9ecd-4200-9c7d-cf3cc20b0aa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:08:31.360818 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca244d95-9ecd-4200-9c7d-cf3cc20b0aa7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:08:31.361531 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca244d95-9ecd-4200-9c7d-cf3cc20b0aa7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:08:31.363463 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca244d95-9ecd-4200-9c7d-cf3cc20b0aa7 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:08:31.393376 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1228/2423] 10.222.0.46 () {60 vars in 1152 bytes} [Thu Jun 23 16:08:31 2022] POST /identity/v3/projects => generated 389 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 16:08:31.394586 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255211ff-7caa-426a-9a17-f74771e2f923 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:08:31.406049 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255211ff-7caa-426a-9a17-f74771e2f923 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:08:31.466305 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b10a4af-f20a-45f8-b53c-056286b0701d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:08:31.467045 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b10a4af-f20a-45f8-b53c-056286b0701d admin admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:08:31.467326 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b10a4af-f20a-45f8-b53c-056286b0701d admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:08:31.467611 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b10a4af-f20a-45f8-b53c-056286b0701d admin admin] PATH_INFO: `/v3/users` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:08:31.468377 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b10a4af-f20a-45f8-b53c-056286b0701d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:08:31.470261 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b10a4af-f20a-45f8-b53c-056286b0701d admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:08:31.491483 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1196/2424] 10.222.0.46 () {60 vars in 1143 bytes} [Thu Jun 23 16:08:31 2022] POST /identity/v3/users => generated 301 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 16:08:31.495248 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca244d95-9ecd-4200-9c7d-cf3cc20b0aa7 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:08:31.506451 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca244d95-9ecd-4200-9c7d-cf3cc20b0aa7 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:08:31.511326 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f223f915-e60e-4dd9-9953-0c61ec9fa084 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:08:31.512046 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f223f915-e60e-4dd9-9953-0c61ec9fa084 admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:08:31.512368 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f223f915-e60e-4dd9-9953-0c61ec9fa084 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:08:31.512646 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f223f915-e60e-4dd9-9953-0c61ec9fa084 admin admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:08:31.513382 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f223f915-e60e-4dd9-9953-0c61ec9fa084 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:08:31.515064 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f223f915-e60e-4dd9-9953-0c61ec9fa084 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:08:31.526052 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1229/2425] 10.222.0.46 () {58 vars in 1121 bytes} [Thu Jun 23 16:08:31 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 16:08:31.530245 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b10a4af-f20a-45f8-b53c-056286b0701d admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:08:31.540399 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b10a4af-f20a-45f8-b53c-056286b0701d admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:08:31.545694 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b40fb2-101c-4597-9164-3cbdeb1d8230 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:08:31.546578 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b40fb2-101c-4597-9164-3cbdeb1d8230 admin admin] REQUEST_METHOD: `PUT` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:08:31.546888 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b40fb2-101c-4597-9164-3cbdeb1d8230 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:08:31.547170 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b40fb2-101c-4597-9164-3cbdeb1d8230 admin admin] PATH_INFO: `/v3/projects/d593501181ce460394e5e0b1aede7397/users/1d655b50b7a44e879239c114bbfd5d7b/roles/812d3c567c52469cb2b35709fbdc857c` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:08:31.550270 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:08:31.566183 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b40fb2-101c-4597-9164-3cbdeb1d8230 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d593501181ce460394e5e0b1aede7397, user_id=1d655b50b7a44e879239c114bbfd5d7b, role_id=812d3c567c52469cb2b35709fbdc857c)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:08:31.568932 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b40fb2-101c-4597-9164-3cbdeb1d8230 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:08:31.582590 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1197/2426] 10.222.0.46 () {60 vars in 1482 bytes} [Thu Jun 23 16:08:31 2022] PUT /identity/v3/projects/d593501181ce460394e5e0b1aede7397/users/1d655b50b7a44e879239c114bbfd5d7b/roles/812d3c567c52469cb2b35709fbdc857c => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 16:08:31.586602 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f223f915-e60e-4dd9-9953-0c61ec9fa084 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:08:31.596748 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f223f915-e60e-4dd9-9953-0c61ec9fa084 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:08:31.608859 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:08:31.656121 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e914a6ab-aeaa-45c7-b12d-c4ba1f84bed5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:08:31.656792 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e914a6ab-aeaa-45c7-b12d-c4ba1f84bed5 admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:08:31.657077 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e914a6ab-aeaa-45c7-b12d-c4ba1f84bed5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:08:31.657351 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e914a6ab-aeaa-45c7-b12d-c4ba1f84bed5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:08:31.658073 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e914a6ab-aeaa-45c7-b12d-c4ba1f84bed5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:08:31.659754 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e914a6ab-aeaa-45c7-b12d-c4ba1f84bed5 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:08:31.670130 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1230/2427] 10.222.0.46 () {58 vars in 1121 bytes} [Thu Jun 23 16:08:31 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 16:08:31.674339 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b40fb2-101c-4597-9164-3cbdeb1d8230 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:08:31.685401 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b40fb2-101c-4597-9164-3cbdeb1d8230 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:08:31.689998 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446a4377-de4a-4f29-b4fc-ce0a8640ea87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:08:31.690952 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446a4377-de4a-4f29-b4fc-ce0a8640ea87 admin admin] REQUEST_METHOD: `PUT` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:08:31.691248 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446a4377-de4a-4f29-b4fc-ce0a8640ea87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:08:31.691529 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446a4377-de4a-4f29-b4fc-ce0a8640ea87 admin admin] PATH_INFO: `/v3/projects/d593501181ce460394e5e0b1aede7397/users/1d655b50b7a44e879239c114bbfd5d7b/roles/812d3c567c52469cb2b35709fbdc857c` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:08:31.693517 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446a4377-de4a-4f29-b4fc-ce0a8640ea87 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d593501181ce460394e5e0b1aede7397, user_id=1d655b50b7a44e879239c114bbfd5d7b, role_id=812d3c567c52469cb2b35709fbdc857c)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:08:31.696331 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446a4377-de4a-4f29-b4fc-ce0a8640ea87 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:08:31.708159 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1198/2428] 10.222.0.46 () {60 vars in 1482 bytes} [Thu Jun 23 16:08:31 2022] PUT /identity/v3/projects/d593501181ce460394e5e0b1aede7397/users/1d655b50b7a44e879239c114bbfd5d7b/roles/812d3c567c52469cb2b35709fbdc857c => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 16:08:31.720378 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3964a7-97be-4062-9c7d-f23ec8dbe770 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:08:31.720667 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3964a7-97be-4062-9c7d-f23ec8dbe770 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:08:31.720954 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3964a7-97be-4062-9c7d-f23ec8dbe770 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:08:31.747918 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-db3964a7-97be-4062-9c7d-f23ec8dbe770 None None] MFA Rules not processed for user `1d655b50b7a44e879239c114bbfd5d7b`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:08:31.749069 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-db3964a7-97be-4062-9c7d-f23ec8dbe770 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:08:31.829378 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1231/2429] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:08:31 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 16:08:31.840728 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446a4377-de4a-4f29-b4fc-ce0a8640ea87 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:08:31.842505 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-446a4377-de4a-4f29-b4fc-ce0a8640ea87 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:08:31.901724 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446a4377-de4a-4f29-b4fc-ce0a8640ea87 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:08:31.905925 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41478846-a248-4566-a285-6c00964d02fd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce18eaaba3a6477684fff1fd98ec8b51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:08:31.906575 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41478846-a248-4566-a285-6c00964d02fd service cinder] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:08:31.906891 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41478846-a248-4566-a285-6c00964d02fd service cinder] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:08:31.907165 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41478846-a248-4566-a285-6c00964d02fd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:08:31.916609 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41478846-a248-4566-a285-6c00964d02fd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:08:31.918037 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41478846-a248-4566-a285-6c00964d02fd service cinder] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:08:31.928098 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1199/2430] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jun 23 16:08:31 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 16:08:32.059539 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3964a7-97be-4062-9c7d-f23ec8dbe770 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:08:32.123370 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3964a7-97be-4062-9c7d-f23ec8dbe770 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:08:32.127859 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630b4b78-5d76-4f99-b274-648cdcbe522e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:08:32.128729 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630b4b78-5d76-4f99-b274-648cdcbe522e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:08:32.129007 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630b4b78-5d76-4f99-b274-648cdcbe522e admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:08:32.129280 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630b4b78-5d76-4f99-b274-648cdcbe522e admin admin] PATH_INFO: `/v3/users/507b4fa5d05148abbeb66f3ea275a73e` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:08:32.130454 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630b4b78-5d76-4f99-b274-648cdcbe522e admin admin] RBAC: Authorizing `identity:delete_user(user_id=507b4fa5d05148abbeb66f3ea275a73e)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:08:32.132604 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630b4b78-5d76-4f99-b274-648cdcbe522e admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:08:32.174768 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:08:32.181952 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-630b4b78-5d76-4f99-b274-648cdcbe522e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '507b4fa5d05148abbeb66f3ea275a73e'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:08:32.190156 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-630b4b78-5d76-4f99-b274-648cdcbe522e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '507b4fa5d05148abbeb66f3ea275a73e'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:08:32.190911 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-630b4b78-5d76-4f99-b274-648cdcbe522e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '507b4fa5d05148abbeb66f3ea275a73e'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:08:32.191555 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-630b4b78-5d76-4f99-b274-648cdcbe522e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '507b4fa5d05148abbeb66f3ea275a73e'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:08:32.203766 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-630b4b78-5d76-4f99-b274-648cdcbe522e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '507b4fa5d05148abbeb66f3ea275a73e'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:08:32.217462 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1232/2431] 10.222.0.46 () {58 vars in 1223 bytes} [Thu Jun 23 16:08:32 2022] DELETE /identity/v3/users/507b4fa5d05148abbeb66f3ea275a73e => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 16:08:32.221938 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86439f5-b6ad-4127-b271-b2f758cabd31 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:08:32.222267 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86439f5-b6ad-4127-b271-b2f758cabd31 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:08:32.222548 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86439f5-b6ad-4127-b271-b2f758cabd31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:08:32.240871 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-a86439f5-b6ad-4127-b271-b2f758cabd31 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:08:32.242210 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-a86439f5-b6ad-4127-b271-b2f758cabd31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:08:32.303642 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1200/2432] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:08:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 16:08:32.313732 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630b4b78-5d76-4f99-b274-648cdcbe522e admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:08:32.315215 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-630b4b78-5d76-4f99-b274-648cdcbe522e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:08:32.373501 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630b4b78-5d76-4f99-b274-648cdcbe522e admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:08:32.377214 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c9566b-e290-4b4d-bd41-4e63b3b1b19c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51784064951a4619ae09b9167fb0fd0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:08:32.377862 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c9566b-e290-4b4d-bd41-4e63b3b1b19c service neutron] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:08:32.378144 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c9566b-e290-4b4d-bd41-4e63b3b1b19c service neutron] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:08:32.378417 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c9566b-e290-4b4d-bd41-4e63b3b1b19c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:08:32.387526 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c9566b-e290-4b4d-bd41-4e63b3b1b19c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:08:32.388961 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c9566b-e290-4b4d-bd41-4e63b3b1b19c service neutron] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:08:32.398665 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1233/2433] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jun 23 16:08:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 16:08:32.667226 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86439f5-b6ad-4127-b271-b2f758cabd31 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:08:32.668747 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-a86439f5-b6ad-4127-b271-b2f758cabd31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:08:32.680794 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86439f5-b6ad-4127-b271-b2f758cabd31 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:08:32.686033 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5755c499-3021-4a85-924c-5656ea7b72f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:08:32.686937 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5755c499-3021-4a85-924c-5656ea7b72f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:08:32.687253 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5755c499-3021-4a85-924c-5656ea7b72f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:08:32.687496 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5755c499-3021-4a85-924c-5656ea7b72f3 admin admin] PATH_INFO: `/v3/projects/17226cb5d2ed43e1829e1399de364969` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:08:32.688677 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5755c499-3021-4a85-924c-5656ea7b72f3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=17226cb5d2ed43e1829e1399de364969)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:08:32.690773 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5755c499-3021-4a85-924c-5656ea7b72f3 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:08:32.741604 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-5755c499-3021-4a85-924c-5656ea7b72f3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '17226cb5d2ed43e1829e1399de364969', 'request_id': 'req-5755c499-3021-4a85-924c-5656ea7b72f3'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:08:32.748593 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-5755c499-3021-4a85-924c-5656ea7b72f3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '17226cb5d2ed43e1829e1399de364969', 'request_id': 'req-5755c499-3021-4a85-924c-5656ea7b72f3'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:08:32.755225 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-5755c499-3021-4a85-924c-5656ea7b72f3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '17226cb5d2ed43e1829e1399de364969', 'request_id': 'req-5755c499-3021-4a85-924c-5656ea7b72f3'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:08:32.762782 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:08:32.768707 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-5755c499-3021-4a85-924c-5656ea7b72f3 admin admin] The token cache is being invalidate because project 17226cb5d2ed43e1829e1399de364969 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80154) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 16:08:32.769354 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-5755c499-3021-4a85-924c-5656ea7b72f3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:08:32.770008 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-5755c499-3021-4a85-924c-5656ea7b72f3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:08:32.771742 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1201/2434] 10.222.0.46 () {58 vars in 1232 bytes} [Thu Jun 23 16:08:32 2022] DELETE /identity/v3/projects/17226cb5d2ed43e1829e1399de364969 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 16:08:32.776183 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c9566b-e290-4b4d-bd41-4e63b3b1b19c service neutron] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:08:32.785944 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c9566b-e290-4b4d-bd41-4e63b3b1b19c service neutron] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:08:32.843733 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770b65a7-342d-4f02-9aa4-a0d74da21b24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:08:32.844616 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770b65a7-342d-4f02-9aa4-a0d74da21b24 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:08:32.844902 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770b65a7-342d-4f02-9aa4-a0d74da21b24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:08:32.845186 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770b65a7-342d-4f02-9aa4-a0d74da21b24 admin admin] PATH_INFO: `/v3/users/1d655b50b7a44e879239c114bbfd5d7b` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:08:32.846390 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770b65a7-342d-4f02-9aa4-a0d74da21b24 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1d655b50b7a44e879239c114bbfd5d7b)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:08:32.848379 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770b65a7-342d-4f02-9aa4-a0d74da21b24 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:08:32.894622 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-770b65a7-342d-4f02-9aa4-a0d74da21b24 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1d655b50b7a44e879239c114bbfd5d7b'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:08:32.901690 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-770b65a7-342d-4f02-9aa4-a0d74da21b24 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1d655b50b7a44e879239c114bbfd5d7b'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:08:32.902348 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-770b65a7-342d-4f02-9aa4-a0d74da21b24 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1d655b50b7a44e879239c114bbfd5d7b'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:08:32.902988 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-770b65a7-342d-4f02-9aa4-a0d74da21b24 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1d655b50b7a44e879239c114bbfd5d7b'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:08:32.907433 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:08:32.916566 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-770b65a7-342d-4f02-9aa4-a0d74da21b24 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1d655b50b7a44e879239c114bbfd5d7b'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:08:32.929877 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1234/2435] 10.222.0.46 () {58 vars in 1223 bytes} [Thu Jun 23 16:08:32 2022] DELETE /identity/v3/users/1d655b50b7a44e879239c114bbfd5d7b => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 16:08:33.223451 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5755c499-3021-4a85-924c-5656ea7b72f3 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:08:33.224988 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-5755c499-3021-4a85-924c-5656ea7b72f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:08:33.244823 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5755c499-3021-4a85-924c-5656ea7b72f3 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:08:33.302396 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf17f49-2c4d-4b66-9c3a-9cc0b9e377f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:08:33.303332 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf17f49-2c4d-4b66-9c3a-9cc0b9e377f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:08:33.303620 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf17f49-2c4d-4b66-9c3a-9cc0b9e377f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:08:33.304111 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf17f49-2c4d-4b66-9c3a-9cc0b9e377f7 admin admin] PATH_INFO: `/v3/projects/d593501181ce460394e5e0b1aede7397` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:08:33.305328 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf17f49-2c4d-4b66-9c3a-9cc0b9e377f7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d593501181ce460394e5e0b1aede7397)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:08:33.307305 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf17f49-2c4d-4b66-9c3a-9cc0b9e377f7 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:08:33.355516 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-5cf17f49-2c4d-4b66-9c3a-9cc0b9e377f7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd593501181ce460394e5e0b1aede7397', 'request_id': 'req-5cf17f49-2c4d-4b66-9c3a-9cc0b9e377f7'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:08:33.360924 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-5cf17f49-2c4d-4b66-9c3a-9cc0b9e377f7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd593501181ce460394e5e0b1aede7397', 'request_id': 'req-5cf17f49-2c4d-4b66-9c3a-9cc0b9e377f7'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:08:33.369102 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-5cf17f49-2c4d-4b66-9c3a-9cc0b9e377f7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd593501181ce460394e5e0b1aede7397', 'request_id': 'req-5cf17f49-2c4d-4b66-9c3a-9cc0b9e377f7'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:08:33.380914 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-5cf17f49-2c4d-4b66-9c3a-9cc0b9e377f7 admin admin] The token cache is being invalidate because project d593501181ce460394e5e0b1aede7397 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80154) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 16:08:33.381213 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-5cf17f49-2c4d-4b66-9c3a-9cc0b9e377f7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:08:33.381860 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-5cf17f49-2c4d-4b66-9c3a-9cc0b9e377f7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:08:33.383451 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1202/2436] 10.222.0.46 () {58 vars in 1232 bytes} [Thu Jun 23 16:08:33 2022] DELETE /identity/v3/projects/d593501181ce460394e5e0b1aede7397 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 16:08:33.389031 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59eab195-1c70-4bd1-9536-afd8df5bfe29 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:08:33.389324 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59eab195-1c70-4bd1-9536-afd8df5bfe29 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:08:33.389601 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59eab195-1c70-4bd1-9536-afd8df5bfe29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:08:33.409354 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-59eab195-1c70-4bd1-9536-afd8df5bfe29 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:08:33.410573 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-59eab195-1c70-4bd1-9536-afd8df5bfe29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:08:33.469420 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1235/2437] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:08:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 16:08:33.477210 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e6d676-4595-4163-9bb3-4a5c78f49a6e None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:08:33.477503 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e6d676-4595-4163-9bb3-4a5c78f49a6e None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:08:33.477790 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e6d676-4595-4163-9bb3-4a5c78f49a6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:08:33.497588 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-84e6d676-4595-4163-9bb3-4a5c78f49a6e None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:08:33.498853 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-84e6d676-4595-4163-9bb3-4a5c78f49a6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:08:33.506765 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1203/2438] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:08:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 16:08:33.510899 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59eab195-1c70-4bd1-9536-afd8df5bfe29 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:08:33.521199 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59eab195-1c70-4bd1-9536-afd8df5bfe29 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:08:33.526315 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5b22d2-c59e-405d-8fa2-fc607079e58d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:08:33.527054 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5b22d2-c59e-405d-8fa2-fc607079e58d admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:08:33.527342 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5b22d2-c59e-405d-8fa2-fc607079e58d admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:08:33.527622 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5b22d2-c59e-405d-8fa2-fc607079e58d admin admin] PATH_INFO: `/v3/domains` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:08:33.528357 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5b22d2-c59e-405d-8fa2-fc607079e58d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 16:08:33.528672 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5b22d2-c59e-405d-8fa2-fc607079e58d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:08:33.530322 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5b22d2-c59e-405d-8fa2-fc607079e58d admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:08:33.542320 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1236/2439] 10.222.0.46 () {58 vars in 1152 bytes} [Thu Jun 23 16:08:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 16:08:33.546904 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e6d676-4595-4163-9bb3-4a5c78f49a6e None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:08:33.558126 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e6d676-4595-4163-9bb3-4a5c78f49a6e None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:08:33.563353 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6ba158-845b-4e16-8158-1cfd832e6bfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:08:33.564031 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6ba158-845b-4e16-8158-1cfd832e6bfa admin admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:08:33.564318 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6ba158-845b-4e16-8158-1cfd832e6bfa admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:08:33.564600 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6ba158-845b-4e16-8158-1cfd832e6bfa admin admin] PATH_INFO: `/v3/projects` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:08:33.565310 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6ba158-845b-4e16-8158-1cfd832e6bfa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:08:33.567248 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6ba158-845b-4e16-8158-1cfd832e6bfa admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:08:33.593645 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1204/2440] 10.222.0.46 () {60 vars in 1152 bytes} [Thu Jun 23 16:08:33 2022] POST /identity/v3/projects => generated 383 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 16:08:33.597940 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5b22d2-c59e-405d-8fa2-fc607079e58d admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:08:33.608454 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5b22d2-c59e-405d-8fa2-fc607079e58d admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:08:33.671668 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa37de2-8423-4a4c-990c-9101573c0b40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:08:33.672422 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa37de2-8423-4a4c-990c-9101573c0b40 admin admin] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:08:33.672706 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa37de2-8423-4a4c-990c-9101573c0b40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:08:33.673010 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa37de2-8423-4a4c-990c-9101573c0b40 admin admin] PATH_INFO: `/v3/users` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:08:33.673683 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa37de2-8423-4a4c-990c-9101573c0b40 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:08:33.676118 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa37de2-8423-4a4c-990c-9101573c0b40 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:08:33.696715 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1237/2441] 10.222.0.46 () {60 vars in 1143 bytes} [Thu Jun 23 16:08:33 2022] POST /identity/v3/users => generated 292 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 16:08:33.700628 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6ba158-845b-4e16-8158-1cfd832e6bfa admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:08:33.711185 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6ba158-845b-4e16-8158-1cfd832e6bfa admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:08:33.715874 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d904b6ab-e258-42eb-ab09-644eadeb75e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:08:33.716536 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d904b6ab-e258-42eb-ab09-644eadeb75e6 admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:08:33.716828 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d904b6ab-e258-42eb-ab09-644eadeb75e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:08:33.717106 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d904b6ab-e258-42eb-ab09-644eadeb75e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:08:33.717831 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d904b6ab-e258-42eb-ab09-644eadeb75e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:08:33.719501 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d904b6ab-e258-42eb-ab09-644eadeb75e6 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:08:33.729773 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1205/2442] 10.222.0.46 () {58 vars in 1121 bytes} [Thu Jun 23 16:08:33 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 16:08:33.734297 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa37de2-8423-4a4c-990c-9101573c0b40 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:08:33.744334 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa37de2-8423-4a4c-990c-9101573c0b40 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:08:33.749702 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3772a4-a5c4-4264-9b27-d5206ca2ec4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:08:33.750411 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3772a4-a5c4-4264-9b27-d5206ca2ec4a admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:08:33.750775 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3772a4-a5c4-4264-9b27-d5206ca2ec4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:08:33.751178 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3772a4-a5c4-4264-9b27-d5206ca2ec4a admin admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:08:33.752033 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3772a4-a5c4-4264-9b27-d5206ca2ec4a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:08:33.753640 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3772a4-a5c4-4264-9b27-d5206ca2ec4a admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:08:33.764383 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1238/2443] 10.222.0.46 () {58 vars in 1121 bytes} [Thu Jun 23 16:08:33 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 16:08:33.768508 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d904b6ab-e258-42eb-ab09-644eadeb75e6 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:08:33.779105 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d904b6ab-e258-42eb-ab09-644eadeb75e6 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:08:33.783833 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbad1c7c-8186-470e-aac9-50224c2f49b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:08:33.784715 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbad1c7c-8186-470e-aac9-50224c2f49b5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:08:33.785003 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbad1c7c-8186-470e-aac9-50224c2f49b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:08:33.785282 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbad1c7c-8186-470e-aac9-50224c2f49b5 admin admin] PATH_INFO: `/v3/projects/6b8a81a888664796a6363126e725df80/users/af3ec4a48f4b4866b721ad03a93f51a5/roles/e2a5bbf3200a4d4091d72bc7ff78c34d` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:08:33.801620 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbad1c7c-8186-470e-aac9-50224c2f49b5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6b8a81a888664796a6363126e725df80, user_id=af3ec4a48f4b4866b721ad03a93f51a5, role_id=e2a5bbf3200a4d4091d72bc7ff78c34d)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:08:33.804382 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbad1c7c-8186-470e-aac9-50224c2f49b5 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:08:33.820174 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1206/2444] 10.222.0.46 () {60 vars in 1482 bytes} [Thu Jun 23 16:08:33 2022] PUT /identity/v3/projects/6b8a81a888664796a6363126e725df80/users/af3ec4a48f4b4866b721ad03a93f51a5/roles/e2a5bbf3200a4d4091d72bc7ff78c34d => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 16:08:33.830961 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca492ad-b1f0-4922-a907-d0393d049ace None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:08:33.831568 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca492ad-b1f0-4922-a907-d0393d049ace None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:08:33.831994 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca492ad-b1f0-4922-a907-d0393d049ace None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:08:33.862318 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-3ca492ad-b1f0-4922-a907-d0393d049ace None None] MFA Rules not processed for user `af3ec4a48f4b4866b721ad03a93f51a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:08:33.864161 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-3ca492ad-b1f0-4922-a907-d0393d049ace None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:08:33.899075 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:DELETE=3,keystone:SELECT=93,keystone:INSERT=4 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:08:33.932524 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1239/2445] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:08:33 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 16:08:33.937653 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbad1c7c-8186-470e-aac9-50224c2f49b5 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:08:33.963898 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:08:34.000214 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbad1c7c-8186-470e-aac9-50224c2f49b5 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:08:34.004805 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c165af0-7da2-47e3-a7fa-3162151cc51d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:08:34.005464 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c165af0-7da2-47e3-a7fa-3162151cc51d admin admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:08:34.005745 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c165af0-7da2-47e3-a7fa-3162151cc51d admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:08:34.006018 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c165af0-7da2-47e3-a7fa-3162151cc51d admin admin] PATH_INFO: `/v3/projects` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:08:34.006743 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c165af0-7da2-47e3-a7fa-3162151cc51d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:08:34.008637 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c165af0-7da2-47e3-a7fa-3162151cc51d admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:08:34.034512 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1207/2446] 10.222.0.46 () {60 vars in 1152 bytes} [Thu Jun 23 16:08:33 2022] POST /identity/v3/projects => generated 381 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 16:08:34.038544 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca492ad-b1f0-4922-a907-d0393d049ace None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:08:34.049276 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca492ad-b1f0-4922-a907-d0393d049ace None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:08:34.106394 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4039b28-d415-46d2-95b0-680c432b0f69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:08:34.107092 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4039b28-d415-46d2-95b0-680c432b0f69 admin admin] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:08:34.107378 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4039b28-d415-46d2-95b0-680c432b0f69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:08:34.107657 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4039b28-d415-46d2-95b0-680c432b0f69 admin admin] PATH_INFO: `/v3/users` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:08:34.108346 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4039b28-d415-46d2-95b0-680c432b0f69 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:08:34.110268 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4039b28-d415-46d2-95b0-680c432b0f69 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:08:34.130961 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1240/2447] 10.222.0.46 () {60 vars in 1143 bytes} [Thu Jun 23 16:08:34 2022] POST /identity/v3/users => generated 297 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 16:08:34.134942 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c165af0-7da2-47e3-a7fa-3162151cc51d admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:08:34.145850 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c165af0-7da2-47e3-a7fa-3162151cc51d admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:08:34.151470 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29313d6b-0f80-4169-932d-5ebbb02196fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:08:34.152173 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29313d6b-0f80-4169-932d-5ebbb02196fb admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:08:34.152463 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29313d6b-0f80-4169-932d-5ebbb02196fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:08:34.152766 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29313d6b-0f80-4169-932d-5ebbb02196fb admin admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:08:34.153500 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29313d6b-0f80-4169-932d-5ebbb02196fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:08:34.155179 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29313d6b-0f80-4169-932d-5ebbb02196fb admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:08:34.165741 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1208/2448] 10.222.0.46 () {58 vars in 1121 bytes} [Thu Jun 23 16:08:34 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 16:08:34.169975 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4039b28-d415-46d2-95b0-680c432b0f69 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:08:34.179965 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4039b28-d415-46d2-95b0-680c432b0f69 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:08:34.185129 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ee0773-27b7-458e-9009-04c0e8326643 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:08:34.186002 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ee0773-27b7-458e-9009-04c0e8326643 admin admin] REQUEST_METHOD: `PUT` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:08:34.186283 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ee0773-27b7-458e-9009-04c0e8326643 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:08:34.186561 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ee0773-27b7-458e-9009-04c0e8326643 admin admin] PATH_INFO: `/v3/projects/7af5b12fde11494a915d753aaf0f5193/users/d209c072a2354507a239e08eaf2b7b91/roles/812d3c567c52469cb2b35709fbdc857c` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:08:34.202430 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ee0773-27b7-458e-9009-04c0e8326643 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7af5b12fde11494a915d753aaf0f5193, user_id=d209c072a2354507a239e08eaf2b7b91, role_id=812d3c567c52469cb2b35709fbdc857c)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:08:34.204992 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ee0773-27b7-458e-9009-04c0e8326643 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:08:34.219272 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1241/2449] 10.222.0.46 () {60 vars in 1482 bytes} [Thu Jun 23 16:08:34 2022] PUT /identity/v3/projects/7af5b12fde11494a915d753aaf0f5193/users/d209c072a2354507a239e08eaf2b7b91/roles/812d3c567c52469cb2b35709fbdc857c => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 16:08:34.223336 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29313d6b-0f80-4169-932d-5ebbb02196fb admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:08:34.234006 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29313d6b-0f80-4169-932d-5ebbb02196fb admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:08:34.294241 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d20fd5-a9e6-452e-8497-3da4ea756b22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:08:34.294957 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d20fd5-a9e6-452e-8497-3da4ea756b22 admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:08:34.295244 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d20fd5-a9e6-452e-8497-3da4ea756b22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:08:34.295519 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d20fd5-a9e6-452e-8497-3da4ea756b22 admin admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:08:34.296252 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d20fd5-a9e6-452e-8497-3da4ea756b22 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:08:34.297856 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d20fd5-a9e6-452e-8497-3da4ea756b22 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:08:34.308043 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1209/2450] 10.222.0.46 () {58 vars in 1121 bytes} [Thu Jun 23 16:08:34 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 16:08:34.312931 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ee0773-27b7-458e-9009-04c0e8326643 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:08:34.326122 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ee0773-27b7-458e-9009-04c0e8326643 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:08:34.330941 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9ae86d-1a10-422a-83a4-0a9053539ece admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:08:34.331846 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9ae86d-1a10-422a-83a4-0a9053539ece admin admin] REQUEST_METHOD: `PUT` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:08:34.332433 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9ae86d-1a10-422a-83a4-0a9053539ece admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:08:34.333046 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9ae86d-1a10-422a-83a4-0a9053539ece admin admin] PATH_INFO: `/v3/projects/7af5b12fde11494a915d753aaf0f5193/users/d209c072a2354507a239e08eaf2b7b91/roles/812d3c567c52469cb2b35709fbdc857c` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:08:34.335185 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9ae86d-1a10-422a-83a4-0a9053539ece admin admin] RBAC: Authorizing `identity:create_grant(project_id=7af5b12fde11494a915d753aaf0f5193, user_id=d209c072a2354507a239e08eaf2b7b91, role_id=812d3c567c52469cb2b35709fbdc857c)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:08:34.337852 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9ae86d-1a10-422a-83a4-0a9053539ece admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:08:34.348635 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1242/2451] 10.222.0.46 () {60 vars in 1482 bytes} [Thu Jun 23 16:08:34 2022] PUT /identity/v3/projects/7af5b12fde11494a915d753aaf0f5193/users/d209c072a2354507a239e08eaf2b7b91/roles/812d3c567c52469cb2b35709fbdc857c => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 16:08:34.360202 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5432f6-36a0-4bd8-8359-27d662eab751 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:08:34.360571 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5432f6-36a0-4bd8-8359-27d662eab751 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:08:34.360919 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5432f6-36a0-4bd8-8359-27d662eab751 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:08:34.388084 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-8f5432f6-36a0-4bd8-8359-27d662eab751 None None] MFA Rules not processed for user `d209c072a2354507a239e08eaf2b7b91`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:08:34.389222 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-8f5432f6-36a0-4bd8-8359-27d662eab751 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:08:34.460048 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1210/2452] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:08:34 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 16:08:34.472422 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9ae86d-1a10-422a-83a4-0a9053539ece admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:08:34.474181 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-eb9ae86d-1a10-422a-83a4-0a9053539ece admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:08:34.532638 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9ae86d-1a10-422a-83a4-0a9053539ece admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:08:34.536376 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa192ff-7669-42bc-822e-59249fcfccfc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce18eaaba3a6477684fff1fd98ec8b51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:08:34.537027 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa192ff-7669-42bc-822e-59249fcfccfc service cinder] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:08:34.537311 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa192ff-7669-42bc-822e-59249fcfccfc service cinder] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:08:34.537595 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa192ff-7669-42bc-822e-59249fcfccfc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:08:34.547409 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa192ff-7669-42bc-822e-59249fcfccfc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:08:34.548853 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa192ff-7669-42bc-822e-59249fcfccfc service cinder] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:08:34.559552 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1243/2453] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jun 23 16:08:34 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 16:08:34.629392 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5432f6-36a0-4bd8-8359-27d662eab751 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:08:34.639010 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5432f6-36a0-4bd8-8359-27d662eab751 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:08:34.642544 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231d7153-d51f-4938-8a0a-c11b60fd2ee8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce18eaaba3a6477684fff1fd98ec8b51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:08:34.643229 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231d7153-d51f-4938-8a0a-c11b60fd2ee8 service cinder] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:08:34.643518 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231d7153-d51f-4938-8a0a-c11b60fd2ee8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:08:34.643795 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231d7153-d51f-4938-8a0a-c11b60fd2ee8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:08:34.661321 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:08:34.703140 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231d7153-d51f-4938-8a0a-c11b60fd2ee8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:08:34.704576 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231d7153-d51f-4938-8a0a-c11b60fd2ee8 service cinder] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:08:34.713526 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1211/2454] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jun 23 16:08:34 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 16:08:34.978147 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa192ff-7669-42bc-822e-59249fcfccfc service cinder] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:08:35.010207 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:08:35.044342 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa192ff-7669-42bc-822e-59249fcfccfc service cinder] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:08:35.048899 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5801d3f5-3391-4084-bf76-979688b66335 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:08:35.049779 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5801d3f5-3391-4084-bf76-979688b66335 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:08:35.050065 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5801d3f5-3391-4084-bf76-979688b66335 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:08:35.050367 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5801d3f5-3391-4084-bf76-979688b66335 admin admin] PATH_INFO: `/v3/users/af3ec4a48f4b4866b721ad03a93f51a5` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:08:35.051573 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5801d3f5-3391-4084-bf76-979688b66335 admin admin] RBAC: Authorizing `identity:delete_user(user_id=af3ec4a48f4b4866b721ad03a93f51a5)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:08:35.053490 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5801d3f5-3391-4084-bf76-979688b66335 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:08:35.099192 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-5801d3f5-3391-4084-bf76-979688b66335 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'af3ec4a48f4b4866b721ad03a93f51a5'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:08:35.106933 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-5801d3f5-3391-4084-bf76-979688b66335 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'af3ec4a48f4b4866b721ad03a93f51a5'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:08:35.107621 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-5801d3f5-3391-4084-bf76-979688b66335 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'af3ec4a48f4b4866b721ad03a93f51a5'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:08:35.108224 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-5801d3f5-3391-4084-bf76-979688b66335 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'af3ec4a48f4b4866b721ad03a93f51a5'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:08:35.119865 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-5801d3f5-3391-4084-bf76-979688b66335 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'af3ec4a48f4b4866b721ad03a93f51a5'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:08:35.132486 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1244/2455] 10.222.0.46 () {58 vars in 1223 bytes} [Thu Jun 23 16:08:34 2022] DELETE /identity/v3/users/af3ec4a48f4b4866b721ad03a93f51a5 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 16:08:35.137158 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b5abe3-8133-42b1-8a36-44172751db65 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:08:35.137926 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b5abe3-8133-42b1-8a36-44172751db65 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:08:35.138288 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b5abe3-8133-42b1-8a36-44172751db65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:08:35.156719 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-47b5abe3-8133-42b1-8a36-44172751db65 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:08:35.157924 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-47b5abe3-8133-42b1-8a36-44172751db65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:08:35.215704 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1212/2456] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:08:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 16:08:35.225264 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5801d3f5-3391-4084-bf76-979688b66335 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:08:35.226666 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-5801d3f5-3391-4084-bf76-979688b66335 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:08:35.284128 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5801d3f5-3391-4084-bf76-979688b66335 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:08:35.287759 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99964bc7-b8af-48b2-8cc8-bbaa065629e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51784064951a4619ae09b9167fb0fd0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:08:35.288410 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99964bc7-b8af-48b2-8cc8-bbaa065629e3 service neutron] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:08:35.288691 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99964bc7-b8af-48b2-8cc8-bbaa065629e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:08:35.288965 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99964bc7-b8af-48b2-8cc8-bbaa065629e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:08:35.297883 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99964bc7-b8af-48b2-8cc8-bbaa065629e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:08:35.299343 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99964bc7-b8af-48b2-8cc8-bbaa065629e3 service neutron] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:08:35.308680 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1245/2457] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jun 23 16:08:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 16:08:35.592166 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b5abe3-8133-42b1-8a36-44172751db65 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:08:35.593760 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-47b5abe3-8133-42b1-8a36-44172751db65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:08:35.611718 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b5abe3-8133-42b1-8a36-44172751db65 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:08:35.616464 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c932e1a-b3b7-4c31-bc17-0d5aa82c1d07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:08:35.617363 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c932e1a-b3b7-4c31-bc17-0d5aa82c1d07 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:08:35.617653 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c932e1a-b3b7-4c31-bc17-0d5aa82c1d07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:08:35.617934 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c932e1a-b3b7-4c31-bc17-0d5aa82c1d07 admin admin] PATH_INFO: `/v3/projects/6b8a81a888664796a6363126e725df80` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:08:35.619202 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c932e1a-b3b7-4c31-bc17-0d5aa82c1d07 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6b8a81a888664796a6363126e725df80)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:08:35.621168 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c932e1a-b3b7-4c31-bc17-0d5aa82c1d07 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:08:35.673810 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-8c932e1a-b3b7-4c31-bc17-0d5aa82c1d07 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6b8a81a888664796a6363126e725df80', 'request_id': 'req-8c932e1a-b3b7-4c31-bc17-0d5aa82c1d07'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:08:35.679381 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-8c932e1a-b3b7-4c31-bc17-0d5aa82c1d07 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6b8a81a888664796a6363126e725df80', 'request_id': 'req-8c932e1a-b3b7-4c31-bc17-0d5aa82c1d07'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:08:35.688494 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-8c932e1a-b3b7-4c31-bc17-0d5aa82c1d07 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6b8a81a888664796a6363126e725df80', 'request_id': 'req-8c932e1a-b3b7-4c31-bc17-0d5aa82c1d07'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:08:35.701143 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-8c932e1a-b3b7-4c31-bc17-0d5aa82c1d07 admin admin] The token cache is being invalidate because project 6b8a81a888664796a6363126e725df80 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80154) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 16:08:35.701552 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-8c932e1a-b3b7-4c31-bc17-0d5aa82c1d07 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:08:35.702240 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-8c932e1a-b3b7-4c31-bc17-0d5aa82c1d07 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:08:35.703927 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1213/2458] 10.222.0.46 () {58 vars in 1232 bytes} [Thu Jun 23 16:08:35 2022] DELETE /identity/v3/projects/6b8a81a888664796a6363126e725df80 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 16:08:35.707863 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99964bc7-b8af-48b2-8cc8-bbaa065629e3 service neutron] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:08:35.717619 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99964bc7-b8af-48b2-8cc8-bbaa065629e3 service neutron] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:08:35.768079 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:08:35.779096 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce79cb9-6771-4551-86ec-3a346f24756f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:08:35.780135 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce79cb9-6771-4551-86ec-3a346f24756f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:08:35.780426 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce79cb9-6771-4551-86ec-3a346f24756f admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:08:35.780863 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce79cb9-6771-4551-86ec-3a346f24756f admin admin] PATH_INFO: `/v3/users/d209c072a2354507a239e08eaf2b7b91` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:08:35.783413 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce79cb9-6771-4551-86ec-3a346f24756f admin admin] RBAC: Authorizing `identity:delete_user(user_id=d209c072a2354507a239e08eaf2b7b91)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:08:35.785347 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce79cb9-6771-4551-86ec-3a346f24756f admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:08:35.832466 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-9ce79cb9-6771-4551-86ec-3a346f24756f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd209c072a2354507a239e08eaf2b7b91'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:08:35.839576 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-9ce79cb9-6771-4551-86ec-3a346f24756f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd209c072a2354507a239e08eaf2b7b91'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:08:35.840262 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-9ce79cb9-6771-4551-86ec-3a346f24756f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd209c072a2354507a239e08eaf2b7b91'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:08:35.840875 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-9ce79cb9-6771-4551-86ec-3a346f24756f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd209c072a2354507a239e08eaf2b7b91'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:08:35.855457 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-9ce79cb9-6771-4551-86ec-3a346f24756f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd209c072a2354507a239e08eaf2b7b91'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:08:35.876245 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1246/2459] 10.222.0.46 () {58 vars in 1223 bytes} [Thu Jun 23 16:08:35 2022] DELETE /identity/v3/users/d209c072a2354507a239e08eaf2b7b91 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 16:08:36.152403 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c932e1a-b3b7-4c31-bc17-0d5aa82c1d07 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:08:36.153867 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-8c932e1a-b3b7-4c31-bc17-0d5aa82c1d07 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:08:36.167486 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c932e1a-b3b7-4c31-bc17-0d5aa82c1d07 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:08:36.209800 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:08:36.228840 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f08450c-d19e-4d23-a891-ae195fe7dfd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:08:36.229685 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f08450c-d19e-4d23-a891-ae195fe7dfd3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:08:36.229971 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f08450c-d19e-4d23-a891-ae195fe7dfd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:08:36.230252 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f08450c-d19e-4d23-a891-ae195fe7dfd3 admin admin] PATH_INFO: `/v3/projects/7af5b12fde11494a915d753aaf0f5193` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:08:36.232039 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f08450c-d19e-4d23-a891-ae195fe7dfd3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7af5b12fde11494a915d753aaf0f5193)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:08:36.233947 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f08450c-d19e-4d23-a891-ae195fe7dfd3 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:08:36.284320 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-8f08450c-d19e-4d23-a891-ae195fe7dfd3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7af5b12fde11494a915d753aaf0f5193', 'request_id': 'req-8f08450c-d19e-4d23-a891-ae195fe7dfd3'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:08:36.290311 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-8f08450c-d19e-4d23-a891-ae195fe7dfd3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7af5b12fde11494a915d753aaf0f5193', 'request_id': 'req-8f08450c-d19e-4d23-a891-ae195fe7dfd3'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:08:36.300084 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-8f08450c-d19e-4d23-a891-ae195fe7dfd3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7af5b12fde11494a915d753aaf0f5193', 'request_id': 'req-8f08450c-d19e-4d23-a891-ae195fe7dfd3'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:08:36.311804 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-8f08450c-d19e-4d23-a891-ae195fe7dfd3 admin admin] The token cache is being invalidate because project 7af5b12fde11494a915d753aaf0f5193 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80154) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 16:08:36.312095 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-8f08450c-d19e-4d23-a891-ae195fe7dfd3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:08:36.312744 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-8f08450c-d19e-4d23-a891-ae195fe7dfd3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:08:36.314295 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1214/2460] 10.222.0.46 () {58 vars in 1232 bytes} [Thu Jun 23 16:08:36 2022] DELETE /identity/v3/projects/7af5b12fde11494a915d753aaf0f5193 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 16:08:36.319256 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368e9e8b-d806-486c-b934-f0dc925121ea None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:08:36.319751 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368e9e8b-d806-486c-b934-f0dc925121ea None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:08:36.319908 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368e9e8b-d806-486c-b934-f0dc925121ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:08:36.339794 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-368e9e8b-d806-486c-b934-f0dc925121ea None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:08:36.341550 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-368e9e8b-d806-486c-b934-f0dc925121ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:08:36.405613 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1247/2461] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:08:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 16:08:36.413417 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c32476-f719-4c55-8d8e-df1070576dd5 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:08:36.413781 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c32476-f719-4c55-8d8e-df1070576dd5 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:08:36.414126 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c32476-f719-4c55-8d8e-df1070576dd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:08:36.433090 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-91c32476-f719-4c55-8d8e-df1070576dd5 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:08:36.434303 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-91c32476-f719-4c55-8d8e-df1070576dd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:08:36.442401 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1215/2462] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:08:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 16:08:36.446551 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368e9e8b-d806-486c-b934-f0dc925121ea None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:08:36.456491 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368e9e8b-d806-486c-b934-f0dc925121ea None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:08:36.461153 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5ba91e-16e0-47d9-acc8-34f51762814c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:08:36.461814 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5ba91e-16e0-47d9-acc8-34f51762814c admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:08:36.462214 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5ba91e-16e0-47d9-acc8-34f51762814c admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:08:36.462506 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5ba91e-16e0-47d9-acc8-34f51762814c admin admin] PATH_INFO: `/v3/domains` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:08:36.463329 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5ba91e-16e0-47d9-acc8-34f51762814c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 16:08:36.463638 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5ba91e-16e0-47d9-acc8-34f51762814c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:08:36.465823 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5ba91e-16e0-47d9-acc8-34f51762814c admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:08:36.477772 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1248/2463] 10.222.0.46 () {58 vars in 1152 bytes} [Thu Jun 23 16:08:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 16:08:36.482775 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c32476-f719-4c55-8d8e-df1070576dd5 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:08:36.492148 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c32476-f719-4c55-8d8e-df1070576dd5 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:08:36.496901 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4705688c-fabd-4af7-aae0-115fa06ba145 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:08:36.497566 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4705688c-fabd-4af7-aae0-115fa06ba145 admin admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:08:36.497850 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4705688c-fabd-4af7-aae0-115fa06ba145 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:08:36.498131 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4705688c-fabd-4af7-aae0-115fa06ba145 admin admin] PATH_INFO: `/v3/projects` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:08:36.498874 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4705688c-fabd-4af7-aae0-115fa06ba145 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:08:36.500789 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4705688c-fabd-4af7-aae0-115fa06ba145 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:08:36.526236 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1216/2464] 10.222.0.46 () {60 vars in 1152 bytes} [Thu Jun 23 16:08:36 2022] POST /identity/v3/projects => generated 385 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 16:08:36.531488 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5ba91e-16e0-47d9-acc8-34f51762814c admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:08:36.541582 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5ba91e-16e0-47d9-acc8-34f51762814c admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:08:36.598939 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:08:36.607413 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ce2bdd-fc5d-407a-adf3-ff87fa5a15d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:08:36.608333 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ce2bdd-fc5d-407a-adf3-ff87fa5a15d8 admin admin] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:08:36.608733 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ce2bdd-fc5d-407a-adf3-ff87fa5a15d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:08:36.609116 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ce2bdd-fc5d-407a-adf3-ff87fa5a15d8 admin admin] PATH_INFO: `/v3/users` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:08:36.609930 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ce2bdd-fc5d-407a-adf3-ff87fa5a15d8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:08:36.612548 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ce2bdd-fc5d-407a-adf3-ff87fa5a15d8 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:08:36.638988 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1249/2465] 10.222.0.46 () {60 vars in 1143 bytes} [Thu Jun 23 16:08:36 2022] POST /identity/v3/users => generated 293 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 16:08:36.640480 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4705688c-fabd-4af7-aae0-115fa06ba145 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:08:36.651792 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4705688c-fabd-4af7-aae0-115fa06ba145 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:08:36.657210 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5728fa3-9132-448a-a80d-508e09fcb170 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:08:36.657885 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5728fa3-9132-448a-a80d-508e09fcb170 admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:08:36.658172 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5728fa3-9132-448a-a80d-508e09fcb170 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:08:36.658450 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5728fa3-9132-448a-a80d-508e09fcb170 admin admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:08:36.659275 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5728fa3-9132-448a-a80d-508e09fcb170 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:08:36.660879 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5728fa3-9132-448a-a80d-508e09fcb170 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:08:36.672306 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1217/2466] 10.222.0.46 () {58 vars in 1121 bytes} [Thu Jun 23 16:08:36 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 16:08:36.676259 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ce2bdd-fc5d-407a-adf3-ff87fa5a15d8 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:08:36.688490 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ce2bdd-fc5d-407a-adf3-ff87fa5a15d8 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:08:36.694452 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f2a23f-a8d9-4c43-ba66-7e886d9e8955 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:08:36.695181 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f2a23f-a8d9-4c43-ba66-7e886d9e8955 admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:08:36.695467 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f2a23f-a8d9-4c43-ba66-7e886d9e8955 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:08:36.695748 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f2a23f-a8d9-4c43-ba66-7e886d9e8955 admin admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:08:36.696529 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f2a23f-a8d9-4c43-ba66-7e886d9e8955 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:08:36.698231 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f2a23f-a8d9-4c43-ba66-7e886d9e8955 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:08:36.712318 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1250/2467] 10.222.0.46 () {58 vars in 1121 bytes} [Thu Jun 23 16:08:36 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 16:08:36.717481 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5728fa3-9132-448a-a80d-508e09fcb170 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:08:36.727653 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5728fa3-9132-448a-a80d-508e09fcb170 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:08:36.732613 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b971824-8474-4eec-934f-f8a3e0253378 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:08:36.733564 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b971824-8474-4eec-934f-f8a3e0253378 admin admin] REQUEST_METHOD: `PUT` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:08:36.733790 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b971824-8474-4eec-934f-f8a3e0253378 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:08:36.734068 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b971824-8474-4eec-934f-f8a3e0253378 admin admin] PATH_INFO: `/v3/projects/a15809dc6338499a8cfbba9195e82412/users/1ddacdbded4a42ca87643325f869e547/roles/e2a5bbf3200a4d4091d72bc7ff78c34d` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:08:36.748532 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b971824-8474-4eec-934f-f8a3e0253378 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a15809dc6338499a8cfbba9195e82412, user_id=1ddacdbded4a42ca87643325f869e547, role_id=e2a5bbf3200a4d4091d72bc7ff78c34d)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:08:36.751272 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b971824-8474-4eec-934f-f8a3e0253378 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:08:36.765126 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1218/2468] 10.222.0.46 () {60 vars in 1482 bytes} [Thu Jun 23 16:08:36 2022] PUT /identity/v3/projects/a15809dc6338499a8cfbba9195e82412/users/1ddacdbded4a42ca87643325f869e547/roles/e2a5bbf3200a4d4091d72bc7ff78c34d => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 16:08:36.776394 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f174d8-1be5-4e3e-80ff-eb7e9546a559 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:08:36.776681 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f174d8-1be5-4e3e-80ff-eb7e9546a559 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:08:36.776953 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f174d8-1be5-4e3e-80ff-eb7e9546a559 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:08:36.804594 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-63f174d8-1be5-4e3e-80ff-eb7e9546a559 None None] MFA Rules not processed for user `1ddacdbded4a42ca87643325f869e547`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:08:36.805992 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-63f174d8-1be5-4e3e-80ff-eb7e9546a559 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:08:36.871493 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1251/2469] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:08:36 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 16:08:36.875891 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b971824-8474-4eec-934f-f8a3e0253378 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:08:36.941016 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b971824-8474-4eec-934f-f8a3e0253378 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:08:36.945798 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd516448-26f2-4fac-a024-5ab0f842b334 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:08:36.946470 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd516448-26f2-4fac-a024-5ab0f842b334 admin admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:08:36.946789 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd516448-26f2-4fac-a024-5ab0f842b334 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:08:36.947070 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd516448-26f2-4fac-a024-5ab0f842b334 admin admin] PATH_INFO: `/v3/projects` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:08:36.947784 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd516448-26f2-4fac-a024-5ab0f842b334 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:08:36.950256 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd516448-26f2-4fac-a024-5ab0f842b334 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:08:36.963323 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:08:36.977867 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1219/2470] 10.222.0.46 () {60 vars in 1152 bytes} [Thu Jun 23 16:08:36 2022] POST /identity/v3/projects => generated 385 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 16:08:36.981989 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f174d8-1be5-4e3e-80ff-eb7e9546a559 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:08:36.992888 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f174d8-1be5-4e3e-80ff-eb7e9546a559 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:08:37.049631 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bbe52c-586c-4646-b620-96abd21bd352 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:08:37.050311 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bbe52c-586c-4646-b620-96abd21bd352 admin admin] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:08:37.050593 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bbe52c-586c-4646-b620-96abd21bd352 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:08:37.050900 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bbe52c-586c-4646-b620-96abd21bd352 admin admin] PATH_INFO: `/v3/users` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:08:37.051642 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24bbe52c-586c-4646-b620-96abd21bd352 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:08:37.053578 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24bbe52c-586c-4646-b620-96abd21bd352 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:08:37.073622 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1252/2471] 10.222.0.46 () {60 vars in 1143 bytes} [Thu Jun 23 16:08:36 2022] POST /identity/v3/users => generated 293 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 16:08:37.078556 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd516448-26f2-4fac-a024-5ab0f842b334 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:08:37.088879 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd516448-26f2-4fac-a024-5ab0f842b334 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:08:37.093527 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13a0531-6728-4ac3-af8f-6be876cacc8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:08:37.094190 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13a0531-6728-4ac3-af8f-6be876cacc8d admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:08:37.094480 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13a0531-6728-4ac3-af8f-6be876cacc8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:08:37.094794 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13a0531-6728-4ac3-af8f-6be876cacc8d admin admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:08:37.095576 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13a0531-6728-4ac3-af8f-6be876cacc8d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:08:37.097174 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13a0531-6728-4ac3-af8f-6be876cacc8d admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:08:37.107642 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1220/2472] 10.222.0.46 () {58 vars in 1121 bytes} [Thu Jun 23 16:08:37 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 16:08:37.111724 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bbe52c-586c-4646-b620-96abd21bd352 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:08:37.122437 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bbe52c-586c-4646-b620-96abd21bd352 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:08:37.127217 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1647f6-ccc2-4946-a74b-fff0cec9033f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:08:37.127875 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1647f6-ccc2-4946-a74b-fff0cec9033f admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:08:37.128214 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1647f6-ccc2-4946-a74b-fff0cec9033f admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:08:37.128491 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1647f6-ccc2-4946-a74b-fff0cec9033f admin admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:08:37.129223 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1647f6-ccc2-4946-a74b-fff0cec9033f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:08:37.130847 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1647f6-ccc2-4946-a74b-fff0cec9033f admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:08:37.141514 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1253/2473] 10.222.0.46 () {58 vars in 1121 bytes} [Thu Jun 23 16:08:37 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 16:08:37.145459 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13a0531-6728-4ac3-af8f-6be876cacc8d admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:08:37.155684 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13a0531-6728-4ac3-af8f-6be876cacc8d admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:08:37.160991 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb2ab9f-a4fa-42d8-8842-f69bf3f744bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:08:37.161885 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb2ab9f-a4fa-42d8-8842-f69bf3f744bd admin admin] REQUEST_METHOD: `PUT` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:08:37.162173 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb2ab9f-a4fa-42d8-8842-f69bf3f744bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:08:37.162454 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb2ab9f-a4fa-42d8-8842-f69bf3f744bd admin admin] PATH_INFO: `/v3/projects/5b3d8fe5db0b48b49485f462b07761ca/users/9ddfd20c9b6247cfab43a472c91efd7c/roles/e2a5bbf3200a4d4091d72bc7ff78c34d` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:08:37.177546 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb2ab9f-a4fa-42d8-8842-f69bf3f744bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b3d8fe5db0b48b49485f462b07761ca, user_id=9ddfd20c9b6247cfab43a472c91efd7c, role_id=e2a5bbf3200a4d4091d72bc7ff78c34d)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:08:37.180621 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb2ab9f-a4fa-42d8-8842-f69bf3f744bd admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:08:37.195068 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1221/2474] 10.222.0.46 () {60 vars in 1482 bytes} [Thu Jun 23 16:08:37 2022] PUT /identity/v3/projects/5b3d8fe5db0b48b49485f462b07761ca/users/9ddfd20c9b6247cfab43a472c91efd7c/roles/e2a5bbf3200a4d4091d72bc7ff78c34d => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 16:08:37.207378 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f503deb-0f1d-46e6-ad99-58595092d9fa None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:08:37.207704 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f503deb-0f1d-46e6-ad99-58595092d9fa None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:08:37.207989 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f503deb-0f1d-46e6-ad99-58595092d9fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:08:37.234056 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-6f503deb-0f1d-46e6-ad99-58595092d9fa None None] MFA Rules not processed for user `9ddfd20c9b6247cfab43a472c91efd7c`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:08:37.235288 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-6f503deb-0f1d-46e6-ad99-58595092d9fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:08:37.268837 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:08:37.302349 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1254/2475] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:08:37 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 16:08:37.306877 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb2ab9f-a4fa-42d8-8842-f69bf3f744bd admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:08:37.369050 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb2ab9f-a4fa-42d8-8842-f69bf3f744bd admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:08:37.374198 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64df3de6-83c9-4b0e-baa7-f3a517f1dd13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:08:37.375040 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64df3de6-83c9-4b0e-baa7-f3a517f1dd13 admin admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:08:37.375331 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64df3de6-83c9-4b0e-baa7-f3a517f1dd13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:08:37.375618 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64df3de6-83c9-4b0e-baa7-f3a517f1dd13 admin admin] PATH_INFO: `/v3/projects` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:08:37.376328 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64df3de6-83c9-4b0e-baa7-f3a517f1dd13 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:08:37.378243 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64df3de6-83c9-4b0e-baa7-f3a517f1dd13 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:08:37.404621 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1222/2476] 10.222.0.46 () {60 vars in 1152 bytes} [Thu Jun 23 16:08:37 2022] POST /identity/v3/projects => generated 385 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 16:08:37.408943 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f503deb-0f1d-46e6-ad99-58595092d9fa None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:08:37.420466 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f503deb-0f1d-46e6-ad99-58595092d9fa None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:08:37.477442 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce50d701-7c5f-4475-995c-f7890912e0bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:08:37.478120 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce50d701-7c5f-4475-995c-f7890912e0bf admin admin] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:08:37.478411 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce50d701-7c5f-4475-995c-f7890912e0bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:08:37.478725 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce50d701-7c5f-4475-995c-f7890912e0bf admin admin] PATH_INFO: `/v3/users` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:08:37.479419 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce50d701-7c5f-4475-995c-f7890912e0bf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:08:37.481878 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce50d701-7c5f-4475-995c-f7890912e0bf admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:08:37.502322 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1255/2477] 10.222.0.46 () {60 vars in 1143 bytes} [Thu Jun 23 16:08:37 2022] POST /identity/v3/users => generated 299 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 16:08:37.506291 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64df3de6-83c9-4b0e-baa7-f3a517f1dd13 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:08:37.517569 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64df3de6-83c9-4b0e-baa7-f3a517f1dd13 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:08:37.522412 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99978c07-37c3-4b4d-b8d3-f2c3e6820b1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:08:37.523327 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99978c07-37c3-4b4d-b8d3-f2c3e6820b1c admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:08:37.523618 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99978c07-37c3-4b4d-b8d3-f2c3e6820b1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:08:37.523894 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99978c07-37c3-4b4d-b8d3-f2c3e6820b1c admin admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:08:37.524634 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99978c07-37c3-4b4d-b8d3-f2c3e6820b1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:08:37.526281 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99978c07-37c3-4b4d-b8d3-f2c3e6820b1c admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:08:37.540945 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1223/2478] 10.222.0.46 () {58 vars in 1121 bytes} [Thu Jun 23 16:08:37 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 16:08:37.545868 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce50d701-7c5f-4475-995c-f7890912e0bf admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:08:37.563290 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce50d701-7c5f-4475-995c-f7890912e0bf admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:08:37.568738 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e6ebe5-c04a-40c6-a99d-36c9ec7643bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:08:37.569864 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e6ebe5-c04a-40c6-a99d-36c9ec7643bb admin admin] REQUEST_METHOD: `PUT` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:08:37.570174 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e6ebe5-c04a-40c6-a99d-36c9ec7643bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:08:37.570485 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e6ebe5-c04a-40c6-a99d-36c9ec7643bb admin admin] PATH_INFO: `/v3/projects/ee4e40c327de431884253d19c6c463af/users/217b0400696b4272b65378e77d3839c3/roles/812d3c567c52469cb2b35709fbdc857c` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:08:37.586430 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e6ebe5-c04a-40c6-a99d-36c9ec7643bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee4e40c327de431884253d19c6c463af, user_id=217b0400696b4272b65378e77d3839c3, role_id=812d3c567c52469cb2b35709fbdc857c)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:08:37.589114 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e6ebe5-c04a-40c6-a99d-36c9ec7643bb admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:08:37.602443 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1256/2479] 10.222.0.46 () {60 vars in 1482 bytes} [Thu Jun 23 16:08:37 2022] PUT /identity/v3/projects/ee4e40c327de431884253d19c6c463af/users/217b0400696b4272b65378e77d3839c3/roles/812d3c567c52469cb2b35709fbdc857c => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 16:08:37.606999 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99978c07-37c3-4b4d-b8d3-f2c3e6820b1c admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:08:37.617763 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99978c07-37c3-4b4d-b8d3-f2c3e6820b1c admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:08:37.675715 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6598ad-09a3-4ae5-b558-c9366cbf9cb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:08:37.676387 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6598ad-09a3-4ae5-b558-c9366cbf9cb6 admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:08:37.676674 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6598ad-09a3-4ae5-b558-c9366cbf9cb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:08:37.676953 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6598ad-09a3-4ae5-b558-c9366cbf9cb6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:08:37.677685 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6598ad-09a3-4ae5-b558-c9366cbf9cb6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:08:37.679347 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6598ad-09a3-4ae5-b558-c9366cbf9cb6 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:08:37.697205 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1224/2480] 10.222.0.46 () {58 vars in 1121 bytes} [Thu Jun 23 16:08:37 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 16:08:37.698344 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e6ebe5-c04a-40c6-a99d-36c9ec7643bb admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:08:37.710169 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e6ebe5-c04a-40c6-a99d-36c9ec7643bb admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:08:37.715737 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b3598a-0948-408c-adde-c3e2690e447d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:08:37.716443 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b3598a-0948-408c-adde-c3e2690e447d admin admin] REQUEST_METHOD: `PUT` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:08:37.716881 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b3598a-0948-408c-adde-c3e2690e447d admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:08:37.717353 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b3598a-0948-408c-adde-c3e2690e447d admin admin] PATH_INFO: `/v3/projects/ee4e40c327de431884253d19c6c463af/users/217b0400696b4272b65378e77d3839c3/roles/812d3c567c52469cb2b35709fbdc857c` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:08:37.719416 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b3598a-0948-408c-adde-c3e2690e447d admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee4e40c327de431884253d19c6c463af, user_id=217b0400696b4272b65378e77d3839c3, role_id=812d3c567c52469cb2b35709fbdc857c)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:08:37.722164 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b3598a-0948-408c-adde-c3e2690e447d admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:08:37.734546 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1257/2481] 10.222.0.46 () {60 vars in 1482 bytes} [Thu Jun 23 16:08:37 2022] PUT /identity/v3/projects/ee4e40c327de431884253d19c6c463af/users/217b0400696b4272b65378e77d3839c3/roles/812d3c567c52469cb2b35709fbdc857c => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 16:08:37.744671 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563eeea4-93e8-4ce7-9894-10bf1b6897c2 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:08:37.744958 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563eeea4-93e8-4ce7-9894-10bf1b6897c2 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:08:37.745234 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563eeea4-93e8-4ce7-9894-10bf1b6897c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:08:37.777179 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-563eeea4-93e8-4ce7-9894-10bf1b6897c2 None None] MFA Rules not processed for user `217b0400696b4272b65378e77d3839c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:08:37.778958 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-563eeea4-93e8-4ce7-9894-10bf1b6897c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:08:37.796551 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:08:37.855445 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1225/2482] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:08:37 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 16:08:37.864851 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b3598a-0948-408c-adde-c3e2690e447d admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:08:37.867094 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-c0b3598a-0948-408c-adde-c3e2690e447d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:08:37.946541 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b3598a-0948-408c-adde-c3e2690e447d admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:08:37.950803 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f857e2-6e90-4085-ba27-2824ff454840 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce18eaaba3a6477684fff1fd98ec8b51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:08:37.951571 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f857e2-6e90-4085-ba27-2824ff454840 service cinder] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:08:37.951941 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f857e2-6e90-4085-ba27-2824ff454840 service cinder] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:08:37.952368 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f857e2-6e90-4085-ba27-2824ff454840 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:08:37.996659 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:08:38.015695 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f857e2-6e90-4085-ba27-2824ff454840 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:08:38.017202 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f857e2-6e90-4085-ba27-2824ff454840 service cinder] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:08:38.027033 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1258/2483] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jun 23 16:08:37 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 16:08:39.507595 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563eeea4-93e8-4ce7-9894-10bf1b6897c2 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:08:39.509057 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-563eeea4-93e8-4ce7-9894-10bf1b6897c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:08:39.521597 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563eeea4-93e8-4ce7-9894-10bf1b6897c2 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:08:39.525444 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1f0a63-75d3-445f-a052-b5f0c7ecbde5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce18eaaba3a6477684fff1fd98ec8b51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:08:39.526092 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1f0a63-75d3-445f-a052-b5f0c7ecbde5 service cinder] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:08:39.526380 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1f0a63-75d3-445f-a052-b5f0c7ecbde5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:08:39.526667 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1f0a63-75d3-445f-a052-b5f0c7ecbde5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:08:39.588095 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1f0a63-75d3-445f-a052-b5f0c7ecbde5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:08:39.589544 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1f0a63-75d3-445f-a052-b5f0c7ecbde5 service cinder] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:08:39.598437 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1226/2484] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jun 23 16:08:39 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 16:08:40.195742 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f857e2-6e90-4085-ba27-2824ff454840 service cinder] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:08:40.204722 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f857e2-6e90-4085-ba27-2824ff454840 service cinder] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:08:40.208778 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddf7516-a63c-4e51-bf14-4e950926c4c1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce18eaaba3a6477684fff1fd98ec8b51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:08:40.209493 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddf7516-a63c-4e51-bf14-4e950926c4c1 service cinder] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:08:40.209811 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddf7516-a63c-4e51-bf14-4e950926c4c1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:08:40.210079 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddf7516-a63c-4e51-bf14-4e950926c4c1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:08:40.221406 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddf7516-a63c-4e51-bf14-4e950926c4c1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:08:40.222919 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddf7516-a63c-4e51-bf14-4e950926c4c1 service cinder] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:08:40.234159 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1259/2485] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jun 23 16:08:40 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 16:08:44.149627 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1f0a63-75d3-445f-a052-b5f0c7ecbde5 service cinder] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:08:44.211489 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1f0a63-75d3-445f-a052-b5f0c7ecbde5 service cinder] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:08:44.217238 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a41e19-45e3-43bd-bf3c-7f2f77349e91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:08:44.218112 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a41e19-45e3-43bd-bf3c-7f2f77349e91 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:08:44.218395 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a41e19-45e3-43bd-bf3c-7f2f77349e91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:08:44.218679 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a41e19-45e3-43bd-bf3c-7f2f77349e91 admin admin] PATH_INFO: `/v3/users/1ddacdbded4a42ca87643325f869e547` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:08:44.220109 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a41e19-45e3-43bd-bf3c-7f2f77349e91 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1ddacdbded4a42ca87643325f869e547)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:08:44.221821 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a41e19-45e3-43bd-bf3c-7f2f77349e91 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:08:44.273219 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-99a41e19-45e3-43bd-bf3c-7f2f77349e91 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1ddacdbded4a42ca87643325f869e547'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:08:44.274611 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-99a41e19-45e3-43bd-bf3c-7f2f77349e91 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1ddacdbded4a42ca87643325f869e547'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:08:44.282097 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:08:44.284550 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-99a41e19-45e3-43bd-bf3c-7f2f77349e91 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1ddacdbded4a42ca87643325f869e547'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:08:44.293086 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-99a41e19-45e3-43bd-bf3c-7f2f77349e91 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1ddacdbded4a42ca87643325f869e547'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:08:44.294117 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-99a41e19-45e3-43bd-bf3c-7f2f77349e91 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1ddacdbded4a42ca87643325f869e547'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:08:44.316047 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1227/2486] 10.222.0.46 () {58 vars in 1223 bytes} [Thu Jun 23 16:08:44 2022] DELETE /identity/v3/users/1ddacdbded4a42ca87643325f869e547 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 16:08:44.320727 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198b60c6-ddb7-45d4-9d34-f51c76579409 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:08:44.321192 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198b60c6-ddb7-45d4-9d34-f51c76579409 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:08:44.321309 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198b60c6-ddb7-45d4-9d34-f51c76579409 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:08:44.339086 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-198b60c6-ddb7-45d4-9d34-f51c76579409 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:08:44.340208 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-198b60c6-ddb7-45d4-9d34-f51c76579409 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:08:44.398550 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1260/2487] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:08:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 16:08:44.410125 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a41e19-45e3-43bd-bf3c-7f2f77349e91 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:08:44.411631 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-99a41e19-45e3-43bd-bf3c-7f2f77349e91 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:08:44.470408 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a41e19-45e3-43bd-bf3c-7f2f77349e91 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:08:44.474038 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993ca948-63f0-49ca-abaa-9194275676e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51784064951a4619ae09b9167fb0fd0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:08:44.474720 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993ca948-63f0-49ca-abaa-9194275676e5 service neutron] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:08:44.475010 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993ca948-63f0-49ca-abaa-9194275676e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:08:44.475289 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993ca948-63f0-49ca-abaa-9194275676e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:08:44.484378 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993ca948-63f0-49ca-abaa-9194275676e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:08:44.485816 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993ca948-63f0-49ca-abaa-9194275676e5 service neutron] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:08:44.495296 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1228/2488] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jun 23 16:08:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 16:08:44.766566 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198b60c6-ddb7-45d4-9d34-f51c76579409 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:08:44.768149 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-198b60c6-ddb7-45d4-9d34-f51c76579409 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:08:44.782870 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198b60c6-ddb7-45d4-9d34-f51c76579409 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:08:44.788247 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6fdfa0-e6ea-48e1-b7ba-210589dc9074 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:08:44.789441 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6fdfa0-e6ea-48e1-b7ba-210589dc9074 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:08:44.789880 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6fdfa0-e6ea-48e1-b7ba-210589dc9074 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:08:44.790387 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6fdfa0-e6ea-48e1-b7ba-210589dc9074 admin admin] PATH_INFO: `/v3/projects/a15809dc6338499a8cfbba9195e82412` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:08:44.792065 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6fdfa0-e6ea-48e1-b7ba-210589dc9074 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a15809dc6338499a8cfbba9195e82412)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:08:44.794621 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6fdfa0-e6ea-48e1-b7ba-210589dc9074 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:08:44.845610 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-ed6fdfa0-e6ea-48e1-b7ba-210589dc9074 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a15809dc6338499a8cfbba9195e82412', 'request_id': 'req-ed6fdfa0-e6ea-48e1-b7ba-210589dc9074'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:08:44.851044 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-ed6fdfa0-e6ea-48e1-b7ba-210589dc9074 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a15809dc6338499a8cfbba9195e82412', 'request_id': 'req-ed6fdfa0-e6ea-48e1-b7ba-210589dc9074'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:08:44.857212 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-ed6fdfa0-e6ea-48e1-b7ba-210589dc9074 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a15809dc6338499a8cfbba9195e82412', 'request_id': 'req-ed6fdfa0-e6ea-48e1-b7ba-210589dc9074'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:08:44.870622 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-ed6fdfa0-e6ea-48e1-b7ba-210589dc9074 admin admin] The token cache is being invalidate because project a15809dc6338499a8cfbba9195e82412 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80153) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 16:08:44.870940 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-ed6fdfa0-e6ea-48e1-b7ba-210589dc9074 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:08:44.872101 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-ed6fdfa0-e6ea-48e1-b7ba-210589dc9074 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:08:44.873688 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1261/2489] 10.222.0.46 () {58 vars in 1232 bytes} [Thu Jun 23 16:08:44 2022] DELETE /identity/v3/projects/a15809dc6338499a8cfbba9195e82412 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 16:08:44.877851 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993ca948-63f0-49ca-abaa-9194275676e5 service neutron] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:08:44.888142 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993ca948-63f0-49ca-abaa-9194275676e5 service neutron] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:08:44.948476 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cac1d4d-1a75-4687-a065-e78e49274277 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:08:44.949383 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cac1d4d-1a75-4687-a065-e78e49274277 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:08:44.949701 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cac1d4d-1a75-4687-a065-e78e49274277 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:08:44.949984 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cac1d4d-1a75-4687-a065-e78e49274277 admin admin] PATH_INFO: `/v3/users/9ddfd20c9b6247cfab43a472c91efd7c` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:08:44.951294 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cac1d4d-1a75-4687-a065-e78e49274277 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9ddfd20c9b6247cfab43a472c91efd7c)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:08:44.953262 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cac1d4d-1a75-4687-a065-e78e49274277 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:08:45.000167 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-7cac1d4d-1a75-4687-a065-e78e49274277 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9ddfd20c9b6247cfab43a472c91efd7c'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:08:45.000821 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-7cac1d4d-1a75-4687-a065-e78e49274277 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9ddfd20c9b6247cfab43a472c91efd7c'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:08:45.008256 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-7cac1d4d-1a75-4687-a065-e78e49274277 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9ddfd20c9b6247cfab43a472c91efd7c'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:08:45.015609 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-7cac1d4d-1a75-4687-a065-e78e49274277 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9ddfd20c9b6247cfab43a472c91efd7c'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:08:45.016273 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-7cac1d4d-1a75-4687-a065-e78e49274277 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9ddfd20c9b6247cfab43a472c91efd7c'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:08:45.026895 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=86,keystone:INSERT=2,keystone:DELETE=12 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:08:45.035310 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1229/2490] 10.222.0.46 () {58 vars in 1223 bytes} [Thu Jun 23 16:08:44 2022] DELETE /identity/v3/users/9ddfd20c9b6247cfab43a472c91efd7c => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 16:08:45.304908 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6fdfa0-e6ea-48e1-b7ba-210589dc9074 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:08:45.306404 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-ed6fdfa0-e6ea-48e1-b7ba-210589dc9074 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:08:45.321418 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6fdfa0-e6ea-48e1-b7ba-210589dc9074 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:08:45.371474 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:08:45.383642 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89acddd1-495e-44ef-b304-5ec16e133fc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:08:45.384476 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89acddd1-495e-44ef-b304-5ec16e133fc7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:08:45.384759 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89acddd1-495e-44ef-b304-5ec16e133fc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:08:45.385036 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89acddd1-495e-44ef-b304-5ec16e133fc7 admin admin] PATH_INFO: `/v3/projects/5b3d8fe5db0b48b49485f462b07761ca` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:08:45.386225 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89acddd1-495e-44ef-b304-5ec16e133fc7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5b3d8fe5db0b48b49485f462b07761ca)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:08:45.388220 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89acddd1-495e-44ef-b304-5ec16e133fc7 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:08:45.436918 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-89acddd1-495e-44ef-b304-5ec16e133fc7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5b3d8fe5db0b48b49485f462b07761ca', 'request_id': 'req-89acddd1-495e-44ef-b304-5ec16e133fc7'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:08:45.442432 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-89acddd1-495e-44ef-b304-5ec16e133fc7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5b3d8fe5db0b48b49485f462b07761ca', 'request_id': 'req-89acddd1-495e-44ef-b304-5ec16e133fc7'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:08:45.449044 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-89acddd1-495e-44ef-b304-5ec16e133fc7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5b3d8fe5db0b48b49485f462b07761ca', 'request_id': 'req-89acddd1-495e-44ef-b304-5ec16e133fc7'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:08:45.461552 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-89acddd1-495e-44ef-b304-5ec16e133fc7 admin admin] The token cache is being invalidate because project 5b3d8fe5db0b48b49485f462b07761ca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80153) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 16:08:45.461839 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-89acddd1-495e-44ef-b304-5ec16e133fc7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:08:45.462448 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-89acddd1-495e-44ef-b304-5ec16e133fc7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:08:45.464002 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1262/2491] 10.222.0.46 () {58 vars in 1232 bytes} [Thu Jun 23 16:08:45 2022] DELETE /identity/v3/projects/5b3d8fe5db0b48b49485f462b07761ca => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 16:08:45.468461 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cac1d4d-1a75-4687-a065-e78e49274277 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:08:45.469847 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-7cac1d4d-1a75-4687-a065-e78e49274277 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:08:45.483226 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cac1d4d-1a75-4687-a065-e78e49274277 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:08:45.539797 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e51e4a-87e2-4484-901f-7d6713a68482 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:08:45.540683 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e51e4a-87e2-4484-901f-7d6713a68482 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:08:45.540964 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e51e4a-87e2-4484-901f-7d6713a68482 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:08:45.541266 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e51e4a-87e2-4484-901f-7d6713a68482 admin admin] PATH_INFO: `/v3/users/217b0400696b4272b65378e77d3839c3` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:08:45.542454 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e51e4a-87e2-4484-901f-7d6713a68482 admin admin] RBAC: Authorizing `identity:delete_user(user_id=217b0400696b4272b65378e77d3839c3)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:08:45.545046 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e51e4a-87e2-4484-901f-7d6713a68482 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:08:45.591475 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-39e51e4a-87e2-4484-901f-7d6713a68482 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '217b0400696b4272b65378e77d3839c3'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:08:45.592220 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-39e51e4a-87e2-4484-901f-7d6713a68482 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '217b0400696b4272b65378e77d3839c3'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:08:45.599120 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-39e51e4a-87e2-4484-901f-7d6713a68482 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '217b0400696b4272b65378e77d3839c3'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:08:45.607272 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-39e51e4a-87e2-4484-901f-7d6713a68482 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '217b0400696b4272b65378e77d3839c3'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:08:45.607906 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-39e51e4a-87e2-4484-901f-7d6713a68482 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '217b0400696b4272b65378e77d3839c3'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:08:45.624878 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1230/2492] 10.222.0.46 () {58 vars in 1223 bytes} [Thu Jun 23 16:08:45 2022] DELETE /identity/v3/users/217b0400696b4272b65378e77d3839c3 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 16:08:45.894673 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89acddd1-495e-44ef-b304-5ec16e133fc7 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:08:45.896211 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-89acddd1-495e-44ef-b304-5ec16e133fc7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:08:45.911655 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89acddd1-495e-44ef-b304-5ec16e133fc7 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:08:45.980229 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbaf319-4f02-4a62-a4da-55200627da5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:08:45.981093 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbaf319-4f02-4a62-a4da-55200627da5d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:08:45.981455 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbaf319-4f02-4a62-a4da-55200627da5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:08:45.981733 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbaf319-4f02-4a62-a4da-55200627da5d admin admin] PATH_INFO: `/v3/projects/ee4e40c327de431884253d19c6c463af` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:08:45.983108 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbaf319-4f02-4a62-a4da-55200627da5d admin admin] RBAC: Authorizing `identity:delete_project(project_id=ee4e40c327de431884253d19c6c463af)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:08:45.985040 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbaf319-4f02-4a62-a4da-55200627da5d admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:08:46.033639 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-6bbaf319-4f02-4a62-a4da-55200627da5d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ee4e40c327de431884253d19c6c463af', 'request_id': 'req-6bbaf319-4f02-4a62-a4da-55200627da5d'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:08:46.043145 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-6bbaf319-4f02-4a62-a4da-55200627da5d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ee4e40c327de431884253d19c6c463af', 'request_id': 'req-6bbaf319-4f02-4a62-a4da-55200627da5d'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:08:46.050064 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-6bbaf319-4f02-4a62-a4da-55200627da5d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ee4e40c327de431884253d19c6c463af', 'request_id': 'req-6bbaf319-4f02-4a62-a4da-55200627da5d'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:08:46.065225 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-6bbaf319-4f02-4a62-a4da-55200627da5d admin admin] The token cache is being invalidate because project ee4e40c327de431884253d19c6c463af was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80153) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 16:08:46.065519 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-6bbaf319-4f02-4a62-a4da-55200627da5d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:08:46.066179 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-6bbaf319-4f02-4a62-a4da-55200627da5d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:08:46.067881 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1263/2493] 10.222.0.46 () {58 vars in 1232 bytes} [Thu Jun 23 16:08:45 2022] DELETE /identity/v3/projects/ee4e40c327de431884253d19c6c463af => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 16:08:46.075262 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0336e3-369a-4932-b6a3-2db8590b8eaf None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:08:46.075559 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0336e3-369a-4932-b6a3-2db8590b8eaf None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:08:46.075848 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0336e3-369a-4932-b6a3-2db8590b8eaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:08:46.094264 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-8c0336e3-369a-4932-b6a3-2db8590b8eaf None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:08:46.095593 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-8c0336e3-369a-4932-b6a3-2db8590b8eaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:08:46.154926 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1231/2494] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:08:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 16:08:46.163921 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e0e74b-12b3-4c29-b52b-7de3fed813ae None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:08:46.164214 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e0e74b-12b3-4c29-b52b-7de3fed813ae None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:08:46.164496 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e0e74b-12b3-4c29-b52b-7de3fed813ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:08:46.182451 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-99e0e74b-12b3-4c29-b52b-7de3fed813ae None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:08:46.183698 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-99e0e74b-12b3-4c29-b52b-7de3fed813ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:08:46.192126 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1264/2495] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:08:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 16:08:46.196304 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0336e3-369a-4932-b6a3-2db8590b8eaf None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:08:46.206938 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0336e3-369a-4932-b6a3-2db8590b8eaf None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:08:46.211811 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590c17dc-8ab8-444b-a3fd-9c53c3bb430c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:08:46.212555 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590c17dc-8ab8-444b-a3fd-9c53c3bb430c admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:08:46.212855 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590c17dc-8ab8-444b-a3fd-9c53c3bb430c admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:08:46.213183 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590c17dc-8ab8-444b-a3fd-9c53c3bb430c admin admin] PATH_INFO: `/v3/domains` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:08:46.214004 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590c17dc-8ab8-444b-a3fd-9c53c3bb430c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 16:08:46.214387 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590c17dc-8ab8-444b-a3fd-9c53c3bb430c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:08:46.216454 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590c17dc-8ab8-444b-a3fd-9c53c3bb430c admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:08:46.225685 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:08:46.229936 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1232/2496] 10.222.0.46 () {58 vars in 1152 bytes} [Thu Jun 23 16:08:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 16:08:46.234280 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e0e74b-12b3-4c29-b52b-7de3fed813ae None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:08:46.239996 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:08:46.244211 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e0e74b-12b3-4c29-b52b-7de3fed813ae None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:08:46.251001 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54dec513-aac0-4bc6-9723-63b0a2ad2b67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:08:46.251866 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54dec513-aac0-4bc6-9723-63b0a2ad2b67 admin admin] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:08:46.252209 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54dec513-aac0-4bc6-9723-63b0a2ad2b67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:08:46.252658 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54dec513-aac0-4bc6-9723-63b0a2ad2b67 admin admin] PATH_INFO: `/v3/projects` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:08:46.253369 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54dec513-aac0-4bc6-9723-63b0a2ad2b67 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:08:46.255471 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54dec513-aac0-4bc6-9723-63b0a2ad2b67 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:08:46.284854 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1265/2497] 10.222.0.46 () {60 vars in 1152 bytes} [Thu Jun 23 16:08:46 2022] POST /identity/v3/projects => generated 375 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 16:08:46.286615 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590c17dc-8ab8-444b-a3fd-9c53c3bb430c admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:08:46.296931 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590c17dc-8ab8-444b-a3fd-9c53c3bb430c admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:08:46.358015 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040115f9-149f-44db-b80a-b232218bcf0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:08:46.358719 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040115f9-149f-44db-b80a-b232218bcf0a admin admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:08:46.359002 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040115f9-149f-44db-b80a-b232218bcf0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:08:46.359280 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040115f9-149f-44db-b80a-b232218bcf0a admin admin] PATH_INFO: `/v3/users` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:08:46.359967 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040115f9-149f-44db-b80a-b232218bcf0a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:08:46.361901 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040115f9-149f-44db-b80a-b232218bcf0a admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:08:46.381089 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1233/2498] 10.222.0.46 () {60 vars in 1143 bytes} [Thu Jun 23 16:08:46 2022] POST /identity/v3/users => generated 288 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 16:08:46.385352 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54dec513-aac0-4bc6-9723-63b0a2ad2b67 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:08:46.396333 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54dec513-aac0-4bc6-9723-63b0a2ad2b67 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:08:46.401302 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b949454-2ce6-4df8-a290-02813fb4dfb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:08:46.401957 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b949454-2ce6-4df8-a290-02813fb4dfb3 admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:08:46.402237 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b949454-2ce6-4df8-a290-02813fb4dfb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:08:46.402511 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b949454-2ce6-4df8-a290-02813fb4dfb3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:08:46.403273 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b949454-2ce6-4df8-a290-02813fb4dfb3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:08:46.404898 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b949454-2ce6-4df8-a290-02813fb4dfb3 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:08:46.415535 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1266/2499] 10.222.0.46 () {58 vars in 1121 bytes} [Thu Jun 23 16:08:46 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 16:08:46.420113 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040115f9-149f-44db-b80a-b232218bcf0a admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:08:46.430195 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040115f9-149f-44db-b80a-b232218bcf0a admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:08:46.435439 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6962b68f-dbf8-4023-8207-27a860282dbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:08:46.436107 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6962b68f-dbf8-4023-8207-27a860282dbf admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:08:46.436389 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6962b68f-dbf8-4023-8207-27a860282dbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:08:46.436667 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6962b68f-dbf8-4023-8207-27a860282dbf admin admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:08:46.437388 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6962b68f-dbf8-4023-8207-27a860282dbf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:08:46.439032 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6962b68f-dbf8-4023-8207-27a860282dbf admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:08:46.449907 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1234/2500] 10.222.0.46 () {58 vars in 1121 bytes} [Thu Jun 23 16:08:46 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 16:08:46.454048 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b949454-2ce6-4df8-a290-02813fb4dfb3 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:08:46.464128 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b949454-2ce6-4df8-a290-02813fb4dfb3 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:08:46.469154 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350ec6ac-6237-468a-bf8f-d86d0f49b687 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:08:46.470065 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350ec6ac-6237-468a-bf8f-d86d0f49b687 admin admin] REQUEST_METHOD: `PUT` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:08:46.470353 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350ec6ac-6237-468a-bf8f-d86d0f49b687 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:08:46.470620 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350ec6ac-6237-468a-bf8f-d86d0f49b687 admin admin] PATH_INFO: `/v3/projects/2e70ca528cc746768ef1b9a5ec72c134/users/3b8523e41d694f9182deedebcc1f9114/roles/e2a5bbf3200a4d4091d72bc7ff78c34d` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:08:46.485859 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350ec6ac-6237-468a-bf8f-d86d0f49b687 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e70ca528cc746768ef1b9a5ec72c134, user_id=3b8523e41d694f9182deedebcc1f9114, role_id=e2a5bbf3200a4d4091d72bc7ff78c34d)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:08:46.488575 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350ec6ac-6237-468a-bf8f-d86d0f49b687 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:08:46.501796 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1267/2501] 10.222.0.46 () {60 vars in 1482 bytes} [Thu Jun 23 16:08:46 2022] PUT /identity/v3/projects/2e70ca528cc746768ef1b9a5ec72c134/users/3b8523e41d694f9182deedebcc1f9114/roles/e2a5bbf3200a4d4091d72bc7ff78c34d => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 16:08:46.511944 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66d2933-c52a-49ce-9845-7369ee1ebac1 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:08:46.512413 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66d2933-c52a-49ce-9845-7369ee1ebac1 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:08:46.512533 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66d2933-c52a-49ce-9845-7369ee1ebac1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:08:46.540189 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-d66d2933-c52a-49ce-9845-7369ee1ebac1 None None] MFA Rules not processed for user `3b8523e41d694f9182deedebcc1f9114`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:08:46.541424 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-d66d2933-c52a-49ce-9845-7369ee1ebac1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:08:46.610186 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1235/2502] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:08:46 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 16:08:46.620853 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350ec6ac-6237-468a-bf8f-d86d0f49b687 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:08:46.623187 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-350ec6ac-6237-468a-bf8f-d86d0f49b687 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:08:46.680957 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350ec6ac-6237-468a-bf8f-d86d0f49b687 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:08:46.685248 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74cc5b4-ac96-48c8-949f-ed31074a5b71 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce18eaaba3a6477684fff1fd98ec8b51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:08:46.685917 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74cc5b4-ac96-48c8-949f-ed31074a5b71 service cinder] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:08:46.686184 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74cc5b4-ac96-48c8-949f-ed31074a5b71 service cinder] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:08:46.686463 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74cc5b4-ac96-48c8-949f-ed31074a5b71 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:08:46.694980 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74cc5b4-ac96-48c8-949f-ed31074a5b71 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:08:46.696464 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74cc5b4-ac96-48c8-949f-ed31074a5b71 service cinder] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:08:46.705261 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1268/2503] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jun 23 16:08:46 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 16:08:56.607312 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=66,keystone:INSERT=3 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:08:56.661240 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66d2933-c52a-49ce-9845-7369ee1ebac1 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:08:56.701889 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=48,keystone:INSERT=2 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:08:56.724253 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66d2933-c52a-49ce-9845-7369ee1ebac1 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:08:56.728840 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234f93bd-0b42-438b-993f-be97257039ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:08:56.729729 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234f93bd-0b42-438b-993f-be97257039ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:08:56.730012 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234f93bd-0b42-438b-993f-be97257039ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:08:56.730291 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234f93bd-0b42-438b-993f-be97257039ee admin admin] PATH_INFO: `/v3/users/3b8523e41d694f9182deedebcc1f9114` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:08:56.731526 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234f93bd-0b42-438b-993f-be97257039ee admin admin] RBAC: Authorizing `identity:delete_user(user_id=3b8523e41d694f9182deedebcc1f9114)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:08:56.733491 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234f93bd-0b42-438b-993f-be97257039ee admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:08:56.786286 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-234f93bd-0b42-438b-993f-be97257039ee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3b8523e41d694f9182deedebcc1f9114'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:08:56.787046 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-234f93bd-0b42-438b-993f-be97257039ee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3b8523e41d694f9182deedebcc1f9114'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:08:56.794030 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-234f93bd-0b42-438b-993f-be97257039ee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3b8523e41d694f9182deedebcc1f9114'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:08:56.801724 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-234f93bd-0b42-438b-993f-be97257039ee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3b8523e41d694f9182deedebcc1f9114'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:08:56.802509 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-234f93bd-0b42-438b-993f-be97257039ee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3b8523e41d694f9182deedebcc1f9114'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:08:56.821605 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1236/2504] 10.222.0.46 () {58 vars in 1223 bytes} [Thu Jun 23 16:08:56 2022] DELETE /identity/v3/users/3b8523e41d694f9182deedebcc1f9114 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 16:08:56.826098 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ceafd6-d3a9-49e1-8767-a48a3f4773b4 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:08:56.827120 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ceafd6-d3a9-49e1-8767-a48a3f4773b4 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:08:56.827406 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ceafd6-d3a9-49e1-8767-a48a3f4773b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:08:56.845052 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-46ceafd6-d3a9-49e1-8767-a48a3f4773b4 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:08:56.846280 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-46ceafd6-d3a9-49e1-8767-a48a3f4773b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:08:56.908092 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1269/2505] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:08:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 16:08:56.917775 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234f93bd-0b42-438b-993f-be97257039ee admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:08:56.919226 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-234f93bd-0b42-438b-993f-be97257039ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:08:56.977246 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234f93bd-0b42-438b-993f-be97257039ee admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:08:56.981085 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b39498b-47f7-44e9-ab75-3e95dcde670f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51784064951a4619ae09b9167fb0fd0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:08:56.981749 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b39498b-47f7-44e9-ab75-3e95dcde670f service neutron] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:08:56.982038 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b39498b-47f7-44e9-ab75-3e95dcde670f service neutron] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:08:56.982877 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b39498b-47f7-44e9-ab75-3e95dcde670f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:08:56.992406 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b39498b-47f7-44e9-ab75-3e95dcde670f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:08:56.993854 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b39498b-47f7-44e9-ab75-3e95dcde670f service neutron] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:08:57.006336 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1237/2506] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jun 23 16:08:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 16:08:57.359384 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ceafd6-d3a9-49e1-8767-a48a3f4773b4 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:08:57.360789 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-46ceafd6-d3a9-49e1-8767-a48a3f4773b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:08:57.372874 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ceafd6-d3a9-49e1-8767-a48a3f4773b4 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:08:57.378889 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188fdd1c-da9f-4291-a7a8-9f6867437154 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:08:57.379772 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188fdd1c-da9f-4291-a7a8-9f6867437154 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:08:57.380054 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188fdd1c-da9f-4291-a7a8-9f6867437154 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:08:57.380353 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188fdd1c-da9f-4291-a7a8-9f6867437154 admin admin] PATH_INFO: `/v3/projects/2e70ca528cc746768ef1b9a5ec72c134` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:08:57.381596 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188fdd1c-da9f-4291-a7a8-9f6867437154 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2e70ca528cc746768ef1b9a5ec72c134)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:08:57.383832 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188fdd1c-da9f-4291-a7a8-9f6867437154 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:08:57.432657 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-188fdd1c-da9f-4291-a7a8-9f6867437154 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2e70ca528cc746768ef1b9a5ec72c134', 'request_id': 'req-188fdd1c-da9f-4291-a7a8-9f6867437154'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:08:57.438245 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-188fdd1c-da9f-4291-a7a8-9f6867437154 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2e70ca528cc746768ef1b9a5ec72c134', 'request_id': 'req-188fdd1c-da9f-4291-a7a8-9f6867437154'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:08:57.444491 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-188fdd1c-da9f-4291-a7a8-9f6867437154 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2e70ca528cc746768ef1b9a5ec72c134', 'request_id': 'req-188fdd1c-da9f-4291-a7a8-9f6867437154'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:08:57.457162 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-188fdd1c-da9f-4291-a7a8-9f6867437154 admin admin] The token cache is being invalidate because project 2e70ca528cc746768ef1b9a5ec72c134 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80153) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 16:08:57.457452 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-188fdd1c-da9f-4291-a7a8-9f6867437154 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:08:57.458081 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-188fdd1c-da9f-4291-a7a8-9f6867437154 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:08:57.459968 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1270/2507] 10.222.0.46 () {58 vars in 1232 bytes} [Thu Jun 23 16:08:57 2022] DELETE /identity/v3/projects/2e70ca528cc746768ef1b9a5ec72c134 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 16:09:07.000104 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:09:07.454203 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:09:26.113772 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b39498b-47f7-44e9-ab75-3e95dcde670f service neutron] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:09:26.115894 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-0b39498b-47f7-44e9-ab75-3e95dcde670f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:09:26.193975 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b39498b-47f7-44e9-ab75-3e95dcde670f service neutron] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:09:26.255453 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c901d39-31fc-473c-b374-5dd87cf9491f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f66988b73644fe38b9849663ee70f25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:09:26.256127 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c901d39-31fc-473c-b374-5dd87cf9491f service placement] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:09:26.256423 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c901d39-31fc-473c-b374-5dd87cf9491f service placement] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:09:26.256703 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c901d39-31fc-473c-b374-5dd87cf9491f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:09:26.258525 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-5c901d39-31fc-473c-b374-5dd87cf9491f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:09:26.270039 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c901d39-31fc-473c-b374-5dd87cf9491f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:09:26.271661 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c901d39-31fc-473c-b374-5dd87cf9491f service placement] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:09:26.281972 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1238/2508] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jun 23 16:09:26 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 16:09:29.757610 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24712bb-8167-4502-80d6-1747750a7885 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:09:29.758136 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24712bb-8167-4502-80d6-1747750a7885 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:09:29.758235 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24712bb-8167-4502-80d6-1747750a7885 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:09:29.776606 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-d24712bb-8167-4502-80d6-1747750a7885 None None] MFA Rules not processed for user `dac5e323b42f41859c85709a2b49623b`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:09:29.777819 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-d24712bb-8167-4502-80d6-1747750a7885 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:09:29.839211 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1271/2509] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:09:29 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 16:09:36.277675 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=58 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:09:39.828015 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=25 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:09:46.941369 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063587bf-a366-47fb-ad5e-b4a49191d4fd None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:09:46.941884 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063587bf-a366-47fb-ad5e-b4a49191d4fd None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:09:46.942002 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063587bf-a366-47fb-ad5e-b4a49191d4fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:09:46.959690 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-063587bf-a366-47fb-ad5e-b4a49191d4fd None None] MFA Rules not processed for user `b423368e84084ac89334f13b723c6b31`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:09:46.960937 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-063587bf-a366-47fb-ad5e-b4a49191d4fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:09:47.014868 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1239/2510] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:09:46 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 16:09:55.278180 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbf32db-eb56-4380-afd1-d332c67a351e None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:09:55.278678 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbf32db-eb56-4380-afd1-d332c67a351e None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:09:55.278814 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbf32db-eb56-4380-afd1-d332c67a351e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:09:55.298303 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-8cbf32db-eb56-4380-afd1-d332c67a351e None None] MFA Rules not processed for user `be48addfaa154555b8d0cb7aac8b32ac`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:09:55.299546 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-8cbf32db-eb56-4380-afd1-d332c67a351e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:09:55.355245 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1272/2511] 10.222.0.46 () {58 vars in 1009 bytes} [Thu Jun 23 16:09:55 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 16:09:55.402071 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063587bf-a366-47fb-ad5e-b4a49191d4fd None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:09:55.403671 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-063587bf-a366-47fb-ad5e-b4a49191d4fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:09:55.475700 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063587bf-a366-47fb-ad5e-b4a49191d4fd None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:09:55.480720 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1587ab6-25aa-4602-9326-ce70facdead0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '242bf02d14724fe4837fcce8730377f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:09:55.481375 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1587ab6-25aa-4602-9326-ce70facdead0 service swift] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:09:55.481662 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1587ab6-25aa-4602-9326-ce70facdead0 service swift] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:09:55.481940 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1587ab6-25aa-4602-9326-ce70facdead0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:09:55.490776 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1587ab6-25aa-4602-9326-ce70facdead0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:09:55.492361 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1587ab6-25aa-4602-9326-ce70facdead0 service swift] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:09:55.501362 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1240/2512] 10.222.0.46 () {60 vars in 1384 bytes} [Thu Jun 23 16:09:55 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 23 16:10:05.348280 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=25 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:10:05.497642 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=57 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:10:09.004834 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbf32db-eb56-4380-afd1-d332c67a351e None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:10:09.006252 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-8cbf32db-eb56-4380-afd1-d332c67a351e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:10:09.062907 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbf32db-eb56-4380-afd1-d332c67a351e None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:10:09.066570 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175197b3-cd07-4c27-8117-bac62d9d24a0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51784064951a4619ae09b9167fb0fd0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:10:09.067257 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175197b3-cd07-4c27-8117-bac62d9d24a0 service neutron] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:10:09.067542 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175197b3-cd07-4c27-8117-bac62d9d24a0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:10:09.067819 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175197b3-cd07-4c27-8117-bac62d9d24a0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:10:09.069610 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-175197b3-cd07-4c27-8117-bac62d9d24a0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:10:09.132752 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175197b3-cd07-4c27-8117-bac62d9d24a0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:10:09.134204 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175197b3-cd07-4c27-8117-bac62d9d24a0 service neutron] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:10:09.144337 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1273/2513] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jun 23 16:10:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 16:10:12.064503 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b685c7-dff2-4145-ad2d-2fb6a60dbedf None None] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:10:12.065012 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b685c7-dff2-4145-ad2d-2fb6a60dbedf None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:10:12.065112 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b685c7-dff2-4145-ad2d-2fb6a60dbedf None None] PATH_INFO: `/` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:10:12.066412 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1241/2514] 10.222.0.46 () {54 vars in 921 bytes} [Thu Jun 23 16:10:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 23 16:10:12.074467 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c075bb-cdbf-40df-a59f-af33be35b73f None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:10:12.075058 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c075bb-cdbf-40df-a59f-af33be35b73f None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:10:12.075269 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c075bb-cdbf-40df-a59f-af33be35b73f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:10:12.119751 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-31c075bb-cdbf-40df-a59f-af33be35b73f None None] MFA Rules not processed for user `705115d19bab424d894bea8b1b48a7f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:10:12.121181 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-31c075bb-cdbf-40df-a59f-af33be35b73f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:10:12.221434 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1274/2515] 10.222.0.46 () {58 vars in 1018 bytes} [Thu Jun 23 16:10:12 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 16:10:12.242455 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b685c7-dff2-4145-ad2d-2fb6a60dbedf None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:10:12.245023 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-75b685c7-dff2-4145-ad2d-2fb6a60dbedf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:10:12.327602 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b685c7-dff2-4145-ad2d-2fb6a60dbedf None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:10:12.331661 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9f2603-b0b7-44bb-80c8-27e6e0717525 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83970271286e451e9008bbf07cd3fa86', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:10:12.332443 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9f2603-b0b7-44bb-80c8-27e6e0717525 service glance] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:10:12.332916 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9f2603-b0b7-44bb-80c8-27e6e0717525 service glance] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:10:12.333397 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9f2603-b0b7-44bb-80c8-27e6e0717525 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:10:12.344959 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9f2603-b0b7-44bb-80c8-27e6e0717525 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:10:12.346525 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9f2603-b0b7-44bb-80c8-27e6e0717525 service glance] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:10:12.357995 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1242/2516] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jun 23 16:10:12 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 16:10:14.615475 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dc2e70-281b-4d67-9fa2-b099a057749f None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:10:14.615952 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dc2e70-281b-4d67-9fa2-b099a057749f None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:10:14.616045 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dc2e70-281b-4d67-9fa2-b099a057749f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:10:14.634328 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-87dc2e70-281b-4d67-9fa2-b099a057749f None None] MFA Rules not processed for user `be48addfaa154555b8d0cb7aac8b32ac`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:10:14.635603 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-87dc2e70-281b-4d67-9fa2-b099a057749f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:10:14.644646 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1275/2517] 10.222.0.46 () {58 vars in 1009 bytes} [Thu Jun 23 16:10:14 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 16:10:14.668062 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9f2603-b0b7-44bb-80c8-27e6e0717525 service glance] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:10:14.684898 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9f2603-b0b7-44bb-80c8-27e6e0717525 service glance] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:10:14.690925 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb0e2b4-8586-44ae-9b48-da32c2f34af6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '242bf02d14724fe4837fcce8730377f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:10:14.691808 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb0e2b4-8586-44ae-9b48-da32c2f34af6 service swift] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:10:14.692087 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb0e2b4-8586-44ae-9b48-da32c2f34af6 service swift] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:10:14.692669 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb0e2b4-8586-44ae-9b48-da32c2f34af6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:10:14.706498 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb0e2b4-8586-44ae-9b48-da32c2f34af6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:10:14.709328 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb0e2b4-8586-44ae-9b48-da32c2f34af6 service swift] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:10:14.720050 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1243/2518] 10.222.0.46 () {60 vars in 1384 bytes} [Thu Jun 23 16:10:14 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 23 16:10:17.024532 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87dc2e70-281b-4d67-9fa2-b099a057749f None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:10:17.025954 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-87dc2e70-281b-4d67-9fa2-b099a057749f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:10:17.078567 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87dc2e70-281b-4d67-9fa2-b099a057749f None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:10:17.097068 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f4b3c5-8796-4b85-83b7-73cd1a3b3ffc service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f66988b73644fe38b9849663ee70f25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:10:17.098113 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f4b3c5-8796-4b85-83b7-73cd1a3b3ffc service placement] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:10:17.099214 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f4b3c5-8796-4b85-83b7-73cd1a3b3ffc service placement] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:10:17.099507 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f4b3c5-8796-4b85-83b7-73cd1a3b3ffc service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:10:17.101477 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-e0f4b3c5-8796-4b85-83b7-73cd1a3b3ffc service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:10:17.126729 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f4b3c5-8796-4b85-83b7-73cd1a3b3ffc service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:10:17.129813 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f4b3c5-8796-4b85-83b7-73cd1a3b3ffc service placement] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:10:17.143161 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1276/2519] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jun 23 16:10:17 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 16:10:17.174185 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb0e2b4-8586-44ae-9b48-da32c2f34af6 service swift] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:10:17.176817 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-2eb0e2b4-8586-44ae-9b48-da32c2f34af6 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:10:17.198454 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb0e2b4-8586-44ae-9b48-da32c2f34af6 service swift] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:10:17.210248 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f5c37a-7b37-4b3d-888d-531a7526a88e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:10:17.211229 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f5c37a-7b37-4b3d-888d-531a7526a88e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:10:17.211743 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f5c37a-7b37-4b3d-888d-531a7526a88e admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:10:17.212115 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f5c37a-7b37-4b3d-888d-531a7526a88e admin admin] PATH_INFO: `/v3/users/2399b4fc64f74475bd8dc7b1634a2c2a` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:10:17.213587 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f5c37a-7b37-4b3d-888d-531a7526a88e admin admin] RBAC: Authorizing `identity:delete_user(user_id=2399b4fc64f74475bd8dc7b1634a2c2a)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:10:17.221308 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f5c37a-7b37-4b3d-888d-531a7526a88e admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:10:17.295966 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-b2f5c37a-7b37-4b3d-888d-531a7526a88e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2399b4fc64f74475bd8dc7b1634a2c2a'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:10:17.296604 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-b2f5c37a-7b37-4b3d-888d-531a7526a88e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2399b4fc64f74475bd8dc7b1634a2c2a'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:10:17.309333 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-b2f5c37a-7b37-4b3d-888d-531a7526a88e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2399b4fc64f74475bd8dc7b1634a2c2a'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:10:17.325488 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-b2f5c37a-7b37-4b3d-888d-531a7526a88e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2399b4fc64f74475bd8dc7b1634a2c2a'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:10:17.326831 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-b2f5c37a-7b37-4b3d-888d-531a7526a88e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2399b4fc64f74475bd8dc7b1634a2c2a'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:10:17.370789 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1244/2520] 10.222.0.46 () {58 vars in 1223 bytes} [Thu Jun 23 16:10:17 2022] DELETE /identity/v3/users/2399b4fc64f74475bd8dc7b1634a2c2a => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 16:10:17.562322 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f4b3c5-8796-4b85-83b7-73cd1a3b3ffc service placement] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:10:17.581606 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f4b3c5-8796-4b85-83b7-73cd1a3b3ffc service placement] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:10:17.604279 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=100 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:10:17.645689 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7b0ff8-f2ed-47d0-a47e-9daf5cf0fd50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:10:17.646658 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7b0ff8-f2ed-47d0-a47e-9daf5cf0fd50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:10:17.647422 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7b0ff8-f2ed-47d0-a47e-9daf5cf0fd50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:10:17.647794 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7b0ff8-f2ed-47d0-a47e-9daf5cf0fd50 admin admin] PATH_INFO: `/v3/projects/bbde90b595244aef82fd4d52d4abfaa8` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:10:17.649301 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7b0ff8-f2ed-47d0-a47e-9daf5cf0fd50 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bbde90b595244aef82fd4d52d4abfaa8)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:10:17.651315 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7b0ff8-f2ed-47d0-a47e-9daf5cf0fd50 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:10:17.702470 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-3d7b0ff8-f2ed-47d0-a47e-9daf5cf0fd50 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bbde90b595244aef82fd4d52d4abfaa8', 'request_id': 'req-3d7b0ff8-f2ed-47d0-a47e-9daf5cf0fd50'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:10:17.708023 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-3d7b0ff8-f2ed-47d0-a47e-9daf5cf0fd50 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bbde90b595244aef82fd4d52d4abfaa8', 'request_id': 'req-3d7b0ff8-f2ed-47d0-a47e-9daf5cf0fd50'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:10:17.714397 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-3d7b0ff8-f2ed-47d0-a47e-9daf5cf0fd50 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bbde90b595244aef82fd4d52d4abfaa8', 'request_id': 'req-3d7b0ff8-f2ed-47d0-a47e-9daf5cf0fd50'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:10:17.729286 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-3d7b0ff8-f2ed-47d0-a47e-9daf5cf0fd50 admin admin] The token cache is being invalidate because project bbde90b595244aef82fd4d52d4abfaa8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80153) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 16:10:17.729598 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-3d7b0ff8-f2ed-47d0-a47e-9daf5cf0fd50 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:10:17.730234 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-3d7b0ff8-f2ed-47d0-a47e-9daf5cf0fd50 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:10:17.732091 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1277/2521] 10.222.0.46 () {58 vars in 1232 bytes} [Thu Jun 23 16:10:17 2022] DELETE /identity/v3/projects/bbde90b595244aef82fd4d52d4abfaa8 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 23 16:10:17.735886 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f5c37a-7b37-4b3d-888d-531a7526a88e admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:10:17.738317 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-b2f5c37a-7b37-4b3d-888d-531a7526a88e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:10:17.754140 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f5c37a-7b37-4b3d-888d-531a7526a88e admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:10:17.829511 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1dc4b3-ac34-46e4-b1fa-27e44ad05500 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:10:17.831120 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1dc4b3-ac34-46e4-b1fa-27e44ad05500 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:10:17.831477 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1dc4b3-ac34-46e4-b1fa-27e44ad05500 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:10:17.831801 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1dc4b3-ac34-46e4-b1fa-27e44ad05500 admin admin] PATH_INFO: `/v3/users/8969b7af3a6d48019b419a5eb4a8846d` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:10:17.833143 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1dc4b3-ac34-46e4-b1fa-27e44ad05500 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8969b7af3a6d48019b419a5eb4a8846d)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:10:17.835944 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1dc4b3-ac34-46e4-b1fa-27e44ad05500 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:10:17.839021 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:10:17.906752 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-db1dc4b3-ac34-46e4-b1fa-27e44ad05500 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8969b7af3a6d48019b419a5eb4a8846d'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:10:17.907414 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-db1dc4b3-ac34-46e4-b1fa-27e44ad05500 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8969b7af3a6d48019b419a5eb4a8846d'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:10:17.914652 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-db1dc4b3-ac34-46e4-b1fa-27e44ad05500 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8969b7af3a6d48019b419a5eb4a8846d'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:10:17.923334 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-db1dc4b3-ac34-46e4-b1fa-27e44ad05500 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8969b7af3a6d48019b419a5eb4a8846d'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:10:17.924365 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-db1dc4b3-ac34-46e4-b1fa-27e44ad05500 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8969b7af3a6d48019b419a5eb4a8846d'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:10:17.947709 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1245/2522] 10.222.0.46 () {58 vars in 1223 bytes} [Thu Jun 23 16:10:17 2022] DELETE /identity/v3/users/8969b7af3a6d48019b419a5eb4a8846d => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 16:10:18.191390 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7b0ff8-f2ed-47d0-a47e-9daf5cf0fd50 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:10:18.193610 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-3d7b0ff8-f2ed-47d0-a47e-9daf5cf0fd50 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:10:18.207793 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7b0ff8-f2ed-47d0-a47e-9daf5cf0fd50 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:10:18.276584 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac0dfc5-a9d4-4e18-a5ed-821d53b34801 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:10:18.277393 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac0dfc5-a9d4-4e18-a5ed-821d53b34801 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:10:18.277685 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac0dfc5-a9d4-4e18-a5ed-821d53b34801 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:10:18.277980 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac0dfc5-a9d4-4e18-a5ed-821d53b34801 admin admin] PATH_INFO: `/v3/projects/5646db2a9be34515a21840c4569b795b` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:10:18.280005 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac0dfc5-a9d4-4e18-a5ed-821d53b34801 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5646db2a9be34515a21840c4569b795b)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:10:18.282162 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac0dfc5-a9d4-4e18-a5ed-821d53b34801 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:10:18.339132 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-fac0dfc5-a9d4-4e18-a5ed-821d53b34801 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5646db2a9be34515a21840c4569b795b', 'request_id': 'req-fac0dfc5-a9d4-4e18-a5ed-821d53b34801'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:10:18.343844 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:10:18.346341 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-fac0dfc5-a9d4-4e18-a5ed-821d53b34801 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5646db2a9be34515a21840c4569b795b', 'request_id': 'req-fac0dfc5-a9d4-4e18-a5ed-821d53b34801'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:10:18.354853 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-fac0dfc5-a9d4-4e18-a5ed-821d53b34801 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5646db2a9be34515a21840c4569b795b', 'request_id': 'req-fac0dfc5-a9d4-4e18-a5ed-821d53b34801'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:10:18.373234 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-fac0dfc5-a9d4-4e18-a5ed-821d53b34801 admin admin] The token cache is being invalidate because project 5646db2a9be34515a21840c4569b795b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80153) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 16:10:18.373535 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-fac0dfc5-a9d4-4e18-a5ed-821d53b34801 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:10:18.374192 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-fac0dfc5-a9d4-4e18-a5ed-821d53b34801 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:10:18.376019 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1278/2523] 10.222.0.46 () {58 vars in 1232 bytes} [Thu Jun 23 16:10:18 2022] DELETE /identity/v3/projects/5646db2a9be34515a21840c4569b795b => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 16:10:27.943426 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=25,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:10:28.369706 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:DELETE=2,keystone:SELECT=5,keystone:INSERT=1 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:10:36.574662 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cd5704-7f2c-4412-8e4d-c9feef6d149e None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:10:36.575166 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cd5704-7f2c-4412-8e4d-c9feef6d149e None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:10:36.575274 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cd5704-7f2c-4412-8e4d-c9feef6d149e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:10:36.594181 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-b5cd5704-7f2c-4412-8e4d-c9feef6d149e None None] MFA Rules not processed for user `be48addfaa154555b8d0cb7aac8b32ac`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:10:36.595424 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-b5cd5704-7f2c-4412-8e4d-c9feef6d149e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:10:36.651218 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1246/2524] 10.222.0.46 () {58 vars in 967 bytes} [Thu Jun 23 16:10:36 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 16:10:36.663445 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac0dfc5-a9d4-4e18-a5ed-821d53b34801 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:10:36.664876 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-fac0dfc5-a9d4-4e18-a5ed-821d53b34801 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:10:36.735688 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac0dfc5-a9d4-4e18-a5ed-821d53b34801 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:10:36.741695 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b442c60-b449-4a6d-8744-d2044a8e2148 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '242bf02d14724fe4837fcce8730377f9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:10:36.742647 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b442c60-b449-4a6d-8744-d2044a8e2148 service swift] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:10:36.743195 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b442c60-b449-4a6d-8744-d2044a8e2148 service swift] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:10:36.743702 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b442c60-b449-4a6d-8744-d2044a8e2148 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:10:36.753558 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b442c60-b449-4a6d-8744-d2044a8e2148 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:10:36.755207 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b442c60-b449-4a6d-8744-d2044a8e2148 service swift] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:10:36.765897 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1279/2525] 10.222.0.46 () {60 vars in 1384 bytes} [Thu Jun 23 16:10:36 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 23 16:10:46.644162 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=25 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:10:46.760772 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=32 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:11:00.206810 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cd5704-7f2c-4412-8e4d-c9feef6d149e None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:11:00.208248 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-b5cd5704-7f2c-4412-8e4d-c9feef6d149e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:11:00.264984 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cd5704-7f2c-4412-8e4d-c9feef6d149e None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:11:00.268722 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ede7818-7801-4844-b8e6-7ec0687d846f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce18eaaba3a6477684fff1fd98ec8b51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:11:00.269370 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ede7818-7801-4844-b8e6-7ec0687d846f service cinder] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:11:00.269650 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ede7818-7801-4844-b8e6-7ec0687d846f service cinder] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:11:00.269920 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ede7818-7801-4844-b8e6-7ec0687d846f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:11:00.271709 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-8ede7818-7801-4844-b8e6-7ec0687d846f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:11:00.328390 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ede7818-7801-4844-b8e6-7ec0687d846f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:11:00.329829 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ede7818-7801-4844-b8e6-7ec0687d846f service cinder] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:11:00.339453 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1247/2526] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jun 23 16:11:00 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 16:11:10.334833 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=48 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:11:11.317716 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f5a787-f275-46f8-b9d4-ca98f649a1cc None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:11:11.318356 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f5a787-f275-46f8-b9d4-ca98f649a1cc None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:11:11.318356 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f5a787-f275-46f8-b9d4-ca98f649a1cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:11:11.336258 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-d7f5a787-f275-46f8-b9d4-ca98f649a1cc None None] MFA Rules not processed for user `c953617a01f1401c966c16467138d628`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:11:11.337659 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-d7f5a787-f275-46f8-b9d4-ca98f649a1cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:11:11.344641 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1280/2527] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:11:11 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 16:11:11.382003 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ede7818-7801-4844-b8e6-7ec0687d846f service cinder] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:11:11.390518 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ede7818-7801-4844-b8e6-7ec0687d846f service cinder] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:11:11.394116 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d00fb35-8bc1-4c45-bd7a-e5630436c673 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce18eaaba3a6477684fff1fd98ec8b51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:11:11.394792 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d00fb35-8bc1-4c45-bd7a-e5630436c673 service cinder] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:11:11.395075 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d00fb35-8bc1-4c45-bd7a-e5630436c673 service cinder] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:11:11.395347 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d00fb35-8bc1-4c45-bd7a-e5630436c673 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:11:11.397132 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-0d00fb35-8bc1-4c45-bd7a-e5630436c673 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:11:11.469574 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d00fb35-8bc1-4c45-bd7a-e5630436c673 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:11:11.470955 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d00fb35-8bc1-4c45-bd7a-e5630436c673 service cinder] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:11:11.482452 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1248/2528] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jun 23 16:11:11 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 16:11:12.225524 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f5a787-f275-46f8-b9d4-ca98f649a1cc None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:11:12.227102 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-d7f5a787-f275-46f8-b9d4-ca98f649a1cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:11:12.288908 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f5a787-f275-46f8-b9d4-ca98f649a1cc None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:11:12.292565 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ecdf29-4e99-45b5-ad85-4de36d4aee68 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51784064951a4619ae09b9167fb0fd0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:11:12.293191 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ecdf29-4e99-45b5-ad85-4de36d4aee68 service neutron] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:11:12.293477 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ecdf29-4e99-45b5-ad85-4de36d4aee68 service neutron] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:11:12.293857 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ecdf29-4e99-45b5-ad85-4de36d4aee68 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:11:12.295670 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-d2ecdf29-4e99-45b5-ad85-4de36d4aee68 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:11:12.309881 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ecdf29-4e99-45b5-ad85-4de36d4aee68 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:11:12.312086 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ecdf29-4e99-45b5-ad85-4de36d4aee68 service neutron] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:11:12.324279 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1281/2529] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jun 23 16:11:12 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 16:11:13.915672 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d00fb35-8bc1-4c45-bd7a-e5630436c673 service cinder] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:11:13.917198 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-0d00fb35-8bc1-4c45-bd7a-e5630436c673 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:11:13.932900 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d00fb35-8bc1-4c45-bd7a-e5630436c673 service cinder] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:11:13.936944 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643ab43c-4c64-4cab-bb1d-c74b747a4ac1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51784064951a4619ae09b9167fb0fd0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:11:13.937613 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643ab43c-4c64-4cab-bb1d-c74b747a4ac1 service neutron] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:11:13.937896 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643ab43c-4c64-4cab-bb1d-c74b747a4ac1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:11:13.938172 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643ab43c-4c64-4cab-bb1d-c74b747a4ac1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:11:13.940075 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-643ab43c-4c64-4cab-bb1d-c74b747a4ac1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:11:13.950460 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643ab43c-4c64-4cab-bb1d-c74b747a4ac1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:11:13.952141 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643ab43c-4c64-4cab-bb1d-c74b747a4ac1 service neutron] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:11:13.962047 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1249/2530] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jun 23 16:11:13 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 16:11:22.319118 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=29 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:11:23.958836 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=40 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:11:30.317541 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ecdf29-4e99-45b5-ad85-4de36d4aee68 service neutron] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:11:30.328303 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ecdf29-4e99-45b5-ad85-4de36d4aee68 service neutron] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:11:30.333057 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0306d6-3802-4e55-93a0-e41a7c4c3f4b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51784064951a4619ae09b9167fb0fd0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:11:30.333859 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0306d6-3802-4e55-93a0-e41a7c4c3f4b service neutron] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:11:30.334385 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0306d6-3802-4e55-93a0-e41a7c4c3f4b service neutron] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:11:30.334765 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0306d6-3802-4e55-93a0-e41a7c4c3f4b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:11:30.337165 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-da0306d6-3802-4e55-93a0-e41a7c4c3f4b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:11:30.421330 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0306d6-3802-4e55-93a0-e41a7c4c3f4b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:11:30.423064 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0306d6-3802-4e55-93a0-e41a7c4c3f4b service neutron] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:11:30.434456 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1282/2531] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jun 23 16:11:30 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 16:11:35.014944 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643ab43c-4c64-4cab-bb1d-c74b747a4ac1 service neutron] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:11:35.023524 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643ab43c-4c64-4cab-bb1d-c74b747a4ac1 service neutron] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:11:35.032944 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d505de-3fa4-41ad-bc7a-a2050bd48da2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51784064951a4619ae09b9167fb0fd0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:11:35.033609 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d505de-3fa4-41ad-bc7a-a2050bd48da2 service neutron] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:11:35.033891 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d505de-3fa4-41ad-bc7a-a2050bd48da2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:11:35.034167 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d505de-3fa4-41ad-bc7a-a2050bd48da2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:11:35.036083 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-b4d505de-3fa4-41ad-bc7a-a2050bd48da2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:11:35.103078 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d505de-3fa4-41ad-bc7a-a2050bd48da2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:11:35.104510 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d505de-3fa4-41ad-bc7a-a2050bd48da2 service neutron] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:11:35.114496 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1250/2532] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jun 23 16:11:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 16:11:38.260107 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0306d6-3802-4e55-93a0-e41a7c4c3f4b service neutron] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:11:38.261699 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-da0306d6-3802-4e55-93a0-e41a7c4c3f4b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:11:38.277684 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0306d6-3802-4e55-93a0-e41a7c4c3f4b service neutron] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:11:38.282018 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a98117-fd4c-49db-921a-5f7095d6ddc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:11:38.282978 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a98117-fd4c-49db-921a-5f7095d6ddc9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:11:38.283378 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a98117-fd4c-49db-921a-5f7095d6ddc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:11:38.283658 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a98117-fd4c-49db-921a-5f7095d6ddc9 admin admin] PATH_INFO: `/v3/users/dac5e323b42f41859c85709a2b49623b` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:11:38.301382 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a98117-fd4c-49db-921a-5f7095d6ddc9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dac5e323b42f41859c85709a2b49623b)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:11:38.303497 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a98117-fd4c-49db-921a-5f7095d6ddc9 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:11:38.348516 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-95a98117-fd4c-49db-921a-5f7095d6ddc9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dac5e323b42f41859c85709a2b49623b'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:11:38.355433 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-95a98117-fd4c-49db-921a-5f7095d6ddc9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dac5e323b42f41859c85709a2b49623b'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:11:38.356097 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-95a98117-fd4c-49db-921a-5f7095d6ddc9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dac5e323b42f41859c85709a2b49623b'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:11:38.356726 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-95a98117-fd4c-49db-921a-5f7095d6ddc9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dac5e323b42f41859c85709a2b49623b'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:11:38.369574 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-95a98117-fd4c-49db-921a-5f7095d6ddc9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dac5e323b42f41859c85709a2b49623b'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:11:38.384686 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1283/2533] 10.222.0.46 () {58 vars in 1223 bytes} [Thu Jun 23 16:11:38 2022] DELETE /identity/v3/users/dac5e323b42f41859c85709a2b49623b => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 16:11:38.563646 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d505de-3fa4-41ad-bc7a-a2050bd48da2 service neutron] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:11:38.580491 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d505de-3fa4-41ad-bc7a-a2050bd48da2 service neutron] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:11:38.638323 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f02b742-04ae-4a15-a451-74093dff786a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:11:38.639239 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f02b742-04ae-4a15-a451-74093dff786a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:11:38.639525 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f02b742-04ae-4a15-a451-74093dff786a admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:11:38.639806 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f02b742-04ae-4a15-a451-74093dff786a admin admin] PATH_INFO: `/v3/projects/2063fb24abb5408794361a9032e56d3f` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:11:38.651963 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f02b742-04ae-4a15-a451-74093dff786a admin admin] RBAC: Authorizing `identity:delete_project(project_id=2063fb24abb5408794361a9032e56d3f)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:11:38.653902 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f02b742-04ae-4a15-a451-74093dff786a admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:11:38.706191 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-5f02b742-04ae-4a15-a451-74093dff786a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2063fb24abb5408794361a9032e56d3f', 'request_id': 'req-5f02b742-04ae-4a15-a451-74093dff786a'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:11:38.715958 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-5f02b742-04ae-4a15-a451-74093dff786a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2063fb24abb5408794361a9032e56d3f', 'request_id': 'req-5f02b742-04ae-4a15-a451-74093dff786a'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:11:38.729998 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-5f02b742-04ae-4a15-a451-74093dff786a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2063fb24abb5408794361a9032e56d3f', 'request_id': 'req-5f02b742-04ae-4a15-a451-74093dff786a'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:11:38.742566 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-5f02b742-04ae-4a15-a451-74093dff786a admin admin] The token cache is being invalidate because project 2063fb24abb5408794361a9032e56d3f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80154) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 16:11:38.742886 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-5f02b742-04ae-4a15-a451-74093dff786a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:11:38.743548 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-5f02b742-04ae-4a15-a451-74093dff786a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:11:38.745106 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1251/2534] 10.222.0.46 () {58 vars in 1232 bytes} [Thu Jun 23 16:11:38 2022] DELETE /identity/v3/projects/2063fb24abb5408794361a9032e56d3f => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 16:11:38.750123 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5b2968-77e6-415f-9d4a-b40f986c975f None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:11:38.750656 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5b2968-77e6-415f-9d4a-b40f986c975f None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:11:38.750981 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5b2968-77e6-415f-9d4a-b40f986c975f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:11:38.770650 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-5d5b2968-77e6-415f-9d4a-b40f986c975f None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:11:38.771952 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-5d5b2968-77e6-415f-9d4a-b40f986c975f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:11:38.836839 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:11:38.854129 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1284/2535] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:11:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 16:11:38.870230 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a18d667-acc5-4a74-a50e-d83c52e41e94 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:11:38.870662 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a18d667-acc5-4a74-a50e-d83c52e41e94 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:11:38.871021 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a18d667-acc5-4a74-a50e-d83c52e41e94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:11:38.889131 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-1a18d667-acc5-4a74-a50e-d83c52e41e94 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:11:38.890359 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-1a18d667-acc5-4a74-a50e-d83c52e41e94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:11:38.899363 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1252/2536] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:11:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 16:11:38.903781 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5b2968-77e6-415f-9d4a-b40f986c975f None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:11:38.914255 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5b2968-77e6-415f-9d4a-b40f986c975f None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:11:38.919438 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8be425-1179-4354-95c2-a18a1778bb65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:11:38.920110 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8be425-1179-4354-95c2-a18a1778bb65 admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:11:38.920391 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8be425-1179-4354-95c2-a18a1778bb65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:11:38.920694 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8be425-1179-4354-95c2-a18a1778bb65 admin admin] PATH_INFO: `/v3/domains` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:11:38.921431 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8be425-1179-4354-95c2-a18a1778bb65 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 16:11:38.921737 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8be425-1179-4354-95c2-a18a1778bb65 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:11:38.923399 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8be425-1179-4354-95c2-a18a1778bb65 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:11:38.936724 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1285/2537] 10.222.0.46 () {58 vars in 1152 bytes} [Thu Jun 23 16:11:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 16:11:38.941526 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a18d667-acc5-4a74-a50e-d83c52e41e94 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:11:38.950857 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a18d667-acc5-4a74-a50e-d83c52e41e94 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:11:38.955516 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47236ccc-fd1f-4362-ab8e-4b0ef0e2ba21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:11:38.956188 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47236ccc-fd1f-4362-ab8e-4b0ef0e2ba21 admin admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:11:38.956967 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47236ccc-fd1f-4362-ab8e-4b0ef0e2ba21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:11:38.957254 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47236ccc-fd1f-4362-ab8e-4b0ef0e2ba21 admin admin] PATH_INFO: `/v3/projects` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:11:38.957946 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47236ccc-fd1f-4362-ab8e-4b0ef0e2ba21 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:11:38.959885 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47236ccc-fd1f-4362-ab8e-4b0ef0e2ba21 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:11:38.964117 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:11:38.990067 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1253/2538] 10.222.0.46 () {60 vars in 1152 bytes} [Thu Jun 23 16:11:38 2022] POST /identity/v3/projects => generated 367 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 16:11:38.993727 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8be425-1179-4354-95c2-a18a1778bb65 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:11:39.004254 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8be425-1179-4354-95c2-a18a1778bb65 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:11:39.061114 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd85691-7400-4451-a9d9-924a42d09209 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:11:39.061790 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd85691-7400-4451-a9d9-924a42d09209 admin admin] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:11:39.062075 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd85691-7400-4451-a9d9-924a42d09209 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:11:39.062357 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd85691-7400-4451-a9d9-924a42d09209 admin admin] PATH_INFO: `/v3/users` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:11:39.063085 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd85691-7400-4451-a9d9-924a42d09209 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:11:39.065003 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd85691-7400-4451-a9d9-924a42d09209 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:11:39.086233 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1286/2539] 10.222.0.46 () {60 vars in 1143 bytes} [Thu Jun 23 16:11:38 2022] POST /identity/v3/users => generated 284 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 16:11:39.090408 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47236ccc-fd1f-4362-ab8e-4b0ef0e2ba21 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:11:39.101211 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47236ccc-fd1f-4362-ab8e-4b0ef0e2ba21 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:11:39.106946 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c3578c-d701-42a1-a71f-75c5a8b34cd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:11:39.107657 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c3578c-d701-42a1-a71f-75c5a8b34cd1 admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:11:39.107946 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c3578c-d701-42a1-a71f-75c5a8b34cd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:11:39.108228 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c3578c-d701-42a1-a71f-75c5a8b34cd1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:11:39.108966 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c3578c-d701-42a1-a71f-75c5a8b34cd1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:11:39.110616 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c3578c-d701-42a1-a71f-75c5a8b34cd1 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:11:39.123504 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1254/2540] 10.222.0.46 () {58 vars in 1121 bytes} [Thu Jun 23 16:11:39 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 16:11:39.128935 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd85691-7400-4451-a9d9-924a42d09209 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:11:39.139075 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd85691-7400-4451-a9d9-924a42d09209 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:11:39.144758 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759c4761-5cf1-4d01-9c4b-9909d923ea8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:11:39.145428 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759c4761-5cf1-4d01-9c4b-9909d923ea8e admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:11:39.146443 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759c4761-5cf1-4d01-9c4b-9909d923ea8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:11:39.146771 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759c4761-5cf1-4d01-9c4b-9909d923ea8e admin admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:11:39.147526 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759c4761-5cf1-4d01-9c4b-9909d923ea8e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:11:39.149439 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759c4761-5cf1-4d01-9c4b-9909d923ea8e admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:11:39.160359 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1287/2541] 10.222.0.46 () {58 vars in 1121 bytes} [Thu Jun 23 16:11:39 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 16:11:39.164658 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c3578c-d701-42a1-a71f-75c5a8b34cd1 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:11:39.174984 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c3578c-d701-42a1-a71f-75c5a8b34cd1 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:11:39.180109 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f256de-1571-4cc7-b7dd-150c10533705 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:11:39.181041 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f256de-1571-4cc7-b7dd-150c10533705 admin admin] REQUEST_METHOD: `PUT` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:11:39.181383 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f256de-1571-4cc7-b7dd-150c10533705 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:11:39.181667 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f256de-1571-4cc7-b7dd-150c10533705 admin admin] PATH_INFO: `/v3/projects/4dc9b6d705724135b8c7e9579baa239c/users/14a481278a2745e6880e00e5cec6ca77/roles/e2a5bbf3200a4d4091d72bc7ff78c34d` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:11:39.198056 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f256de-1571-4cc7-b7dd-150c10533705 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4dc9b6d705724135b8c7e9579baa239c, user_id=14a481278a2745e6880e00e5cec6ca77, role_id=e2a5bbf3200a4d4091d72bc7ff78c34d)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:11:39.200829 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f256de-1571-4cc7-b7dd-150c10533705 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:11:39.215821 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1255/2542] 10.222.0.46 () {60 vars in 1482 bytes} [Thu Jun 23 16:11:39 2022] PUT /identity/v3/projects/4dc9b6d705724135b8c7e9579baa239c/users/14a481278a2745e6880e00e5cec6ca77/roles/e2a5bbf3200a4d4091d72bc7ff78c34d => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 16:11:39.225941 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed78e40-e4c0-4b96-8014-3653e3f2d8d2 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:11:39.226231 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed78e40-e4c0-4b96-8014-3653e3f2d8d2 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:11:39.226511 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed78e40-e4c0-4b96-8014-3653e3f2d8d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:11:39.254168 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-eed78e40-e4c0-4b96-8014-3653e3f2d8d2 None None] MFA Rules not processed for user `14a481278a2745e6880e00e5cec6ca77`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:11:39.255529 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-eed78e40-e4c0-4b96-8014-3653e3f2d8d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:11:39.323265 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1288/2543] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:11:39 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 16:11:39.328467 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f256de-1571-4cc7-b7dd-150c10533705 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:11:39.400036 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f256de-1571-4cc7-b7dd-150c10533705 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:11:39.406520 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39e4003-c06c-444e-815e-1be70ba198c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:11:39.407418 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39e4003-c06c-444e-815e-1be70ba198c0 admin admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:11:39.407711 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39e4003-c06c-444e-815e-1be70ba198c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:11:39.408002 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39e4003-c06c-444e-815e-1be70ba198c0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:11:39.408745 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39e4003-c06c-444e-815e-1be70ba198c0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:11:39.411104 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39e4003-c06c-444e-815e-1be70ba198c0 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:11:39.451549 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1256/2544] 10.222.0.46 () {60 vars in 1152 bytes} [Thu Jun 23 16:11:39 2022] POST /identity/v3/projects => generated 369 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 16:11:39.452203 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed78e40-e4c0-4b96-8014-3653e3f2d8d2 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:11:39.465274 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed78e40-e4c0-4b96-8014-3653e3f2d8d2 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:11:39.527400 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:11:39.540050 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06caf6e-c765-4c91-bd38-3c2aa4cf0e18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:11:39.540842 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06caf6e-c765-4c91-bd38-3c2aa4cf0e18 admin admin] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:11:39.541208 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06caf6e-c765-4c91-bd38-3c2aa4cf0e18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:11:39.541493 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06caf6e-c765-4c91-bd38-3c2aa4cf0e18 admin admin] PATH_INFO: `/v3/users` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:11:39.542203 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06caf6e-c765-4c91-bd38-3c2aa4cf0e18 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:11:39.544538 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06caf6e-c765-4c91-bd38-3c2aa4cf0e18 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:11:39.569043 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1289/2545] 10.222.0.46 () {60 vars in 1143 bytes} [Thu Jun 23 16:11:39 2022] POST /identity/v3/users => generated 291 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 16:11:39.572976 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39e4003-c06c-444e-815e-1be70ba198c0 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:11:39.584871 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39e4003-c06c-444e-815e-1be70ba198c0 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:11:39.589636 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd85411-4f98-4b45-b591-63c8a5250088 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:11:39.590314 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd85411-4f98-4b45-b591-63c8a5250088 admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:11:39.590601 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd85411-4f98-4b45-b591-63c8a5250088 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:11:39.590922 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd85411-4f98-4b45-b591-63c8a5250088 admin admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:11:39.591654 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd85411-4f98-4b45-b591-63c8a5250088 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:11:39.593402 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd85411-4f98-4b45-b591-63c8a5250088 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:11:39.606437 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1257/2546] 10.222.0.46 () {58 vars in 1121 bytes} [Thu Jun 23 16:11:39 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 16:11:39.610712 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06caf6e-c765-4c91-bd38-3c2aa4cf0e18 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:11:39.621556 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06caf6e-c765-4c91-bd38-3c2aa4cf0e18 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:11:39.626297 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07692b6-bcb4-4683-b98d-5ba429c6a4ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:11:39.627215 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07692b6-bcb4-4683-b98d-5ba429c6a4ac admin admin] REQUEST_METHOD: `PUT` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:11:39.627493 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07692b6-bcb4-4683-b98d-5ba429c6a4ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:11:39.627775 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07692b6-bcb4-4683-b98d-5ba429c6a4ac admin admin] PATH_INFO: `/v3/projects/0609f197e8424bdfbaa9a2e501de879b/users/97b7e4e0a8ac4c92844b08beb74d2844/roles/812d3c567c52469cb2b35709fbdc857c` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:11:39.643397 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07692b6-bcb4-4683-b98d-5ba429c6a4ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=0609f197e8424bdfbaa9a2e501de879b, user_id=97b7e4e0a8ac4c92844b08beb74d2844, role_id=812d3c567c52469cb2b35709fbdc857c)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:11:39.646076 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07692b6-bcb4-4683-b98d-5ba429c6a4ac admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:11:39.663663 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1290/2547] 10.222.0.46 () {60 vars in 1482 bytes} [Thu Jun 23 16:11:39 2022] PUT /identity/v3/projects/0609f197e8424bdfbaa9a2e501de879b/users/97b7e4e0a8ac4c92844b08beb74d2844/roles/812d3c567c52469cb2b35709fbdc857c => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 16:11:39.667866 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd85411-4f98-4b45-b591-63c8a5250088 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:11:39.678592 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd85411-4f98-4b45-b591-63c8a5250088 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:11:39.735268 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6d4b60-6696-41f6-b642-275d7e5be255 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:11:39.736070 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6d4b60-6696-41f6-b642-275d7e5be255 admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:11:39.736443 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6d4b60-6696-41f6-b642-275d7e5be255 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:11:39.736753 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6d4b60-6696-41f6-b642-275d7e5be255 admin admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:11:39.737491 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6d4b60-6696-41f6-b642-275d7e5be255 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:11:39.739641 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6d4b60-6696-41f6-b642-275d7e5be255 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:11:39.749952 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1258/2548] 10.222.0.46 () {58 vars in 1121 bytes} [Thu Jun 23 16:11:39 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 16:11:39.754915 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07692b6-bcb4-4683-b98d-5ba429c6a4ac admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:11:39.765811 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07692b6-bcb4-4683-b98d-5ba429c6a4ac admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:11:39.770654 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a619f1d6-833d-45b6-a18a-e1ace9d3393e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:11:39.771577 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a619f1d6-833d-45b6-a18a-e1ace9d3393e admin admin] REQUEST_METHOD: `PUT` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:11:39.771866 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a619f1d6-833d-45b6-a18a-e1ace9d3393e admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:11:39.772147 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a619f1d6-833d-45b6-a18a-e1ace9d3393e admin admin] PATH_INFO: `/v3/projects/0609f197e8424bdfbaa9a2e501de879b/users/97b7e4e0a8ac4c92844b08beb74d2844/roles/812d3c567c52469cb2b35709fbdc857c` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:11:39.774413 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a619f1d6-833d-45b6-a18a-e1ace9d3393e admin admin] RBAC: Authorizing `identity:create_grant(project_id=0609f197e8424bdfbaa9a2e501de879b, user_id=97b7e4e0a8ac4c92844b08beb74d2844, role_id=812d3c567c52469cb2b35709fbdc857c)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:11:39.777100 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a619f1d6-833d-45b6-a18a-e1ace9d3393e admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:11:39.788124 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1291/2549] 10.222.0.46 () {60 vars in 1482 bytes} [Thu Jun 23 16:11:39 2022] PUT /identity/v3/projects/0609f197e8424bdfbaa9a2e501de879b/users/97b7e4e0a8ac4c92844b08beb74d2844/roles/812d3c567c52469cb2b35709fbdc857c => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 16:11:39.799569 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21311f25-2a3f-4cde-af23-3d231507e3c4 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:11:39.799900 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21311f25-2a3f-4cde-af23-3d231507e3c4 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:11:39.800181 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21311f25-2a3f-4cde-af23-3d231507e3c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:11:39.826466 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-21311f25-2a3f-4cde-af23-3d231507e3c4 None None] MFA Rules not processed for user `97b7e4e0a8ac4c92844b08beb74d2844`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:11:39.827710 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-21311f25-2a3f-4cde-af23-3d231507e3c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:11:39.836965 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:11:39.897727 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1259/2550] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:11:39 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 16:11:39.909758 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a619f1d6-833d-45b6-a18a-e1ace9d3393e admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:11:39.911510 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-a619f1d6-833d-45b6-a18a-e1ace9d3393e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:11:39.971361 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a619f1d6-833d-45b6-a18a-e1ace9d3393e admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:11:39.975169 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32367727-fe7f-40d6-9b73-c6885446611c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce18eaaba3a6477684fff1fd98ec8b51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:11:39.975820 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32367727-fe7f-40d6-9b73-c6885446611c service cinder] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:11:39.976113 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32367727-fe7f-40d6-9b73-c6885446611c service cinder] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:11:39.976390 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32367727-fe7f-40d6-9b73-c6885446611c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:11:39.985736 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32367727-fe7f-40d6-9b73-c6885446611c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:11:39.987131 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32367727-fe7f-40d6-9b73-c6885446611c service cinder] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:11:39.997187 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1292/2551] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jun 23 16:11:39 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 16:11:40.041984 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21311f25-2a3f-4cde-af23-3d231507e3c4 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:11:40.043406 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-21311f25-2a3f-4cde-af23-3d231507e3c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:11:40.056400 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21311f25-2a3f-4cde-af23-3d231507e3c4 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:11:40.060104 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52189be2-99fa-437d-9d51-b90bf4f495f0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce18eaaba3a6477684fff1fd98ec8b51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:11:40.060751 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52189be2-99fa-437d-9d51-b90bf4f495f0 service cinder] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:11:40.061040 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52189be2-99fa-437d-9d51-b90bf4f495f0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:11:40.061318 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52189be2-99fa-437d-9d51-b90bf4f495f0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:11:40.118076 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52189be2-99fa-437d-9d51-b90bf4f495f0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:11:40.119540 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52189be2-99fa-437d-9d51-b90bf4f495f0 service cinder] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:11:40.128989 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1260/2552] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jun 23 16:11:40 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 16:11:43.412746 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32367727-fe7f-40d6-9b73-c6885446611c service cinder] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:11:43.474989 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32367727-fe7f-40d6-9b73-c6885446611c service cinder] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:11:43.479576 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f714eab-4fb2-422a-b1df-3747053e432a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:11:43.480448 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f714eab-4fb2-422a-b1df-3747053e432a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:11:43.480730 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f714eab-4fb2-422a-b1df-3747053e432a admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:11:43.481007 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f714eab-4fb2-422a-b1df-3747053e432a admin admin] PATH_INFO: `/v3/users/14a481278a2745e6880e00e5cec6ca77` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:11:43.482185 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f714eab-4fb2-422a-b1df-3747053e432a admin admin] RBAC: Authorizing `identity:delete_user(user_id=14a481278a2745e6880e00e5cec6ca77)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:11:43.484138 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f714eab-4fb2-422a-b1df-3747053e432a admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:11:43.531774 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-8f714eab-4fb2-422a-b1df-3747053e432a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '14a481278a2745e6880e00e5cec6ca77'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:11:43.532989 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:11:43.542261 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-8f714eab-4fb2-422a-b1df-3747053e432a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '14a481278a2745e6880e00e5cec6ca77'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:11:43.543014 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-8f714eab-4fb2-422a-b1df-3747053e432a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '14a481278a2745e6880e00e5cec6ca77'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:11:43.543798 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-8f714eab-4fb2-422a-b1df-3747053e432a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '14a481278a2745e6880e00e5cec6ca77'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:11:43.557345 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-8f714eab-4fb2-422a-b1df-3747053e432a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '14a481278a2745e6880e00e5cec6ca77'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:11:43.571064 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1293/2553] 10.222.0.46 () {58 vars in 1223 bytes} [Thu Jun 23 16:11:43 2022] DELETE /identity/v3/users/14a481278a2745e6880e00e5cec6ca77 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 16:11:43.575621 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad1e3cb-a899-4786-bef9-ce5303269689 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:11:43.576142 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad1e3cb-a899-4786-bef9-ce5303269689 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:11:43.576266 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad1e3cb-a899-4786-bef9-ce5303269689 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:11:43.592825 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-0ad1e3cb-a899-4786-bef9-ce5303269689 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:11:43.594025 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-0ad1e3cb-a899-4786-bef9-ce5303269689 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:11:43.654356 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1261/2554] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:11:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 16:11:43.664456 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f714eab-4fb2-422a-b1df-3747053e432a admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:11:43.665853 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-8f714eab-4fb2-422a-b1df-3747053e432a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:11:43.721998 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f714eab-4fb2-422a-b1df-3747053e432a admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:11:43.726225 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769245c2-434e-4729-a415-587225cd6b61 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51784064951a4619ae09b9167fb0fd0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:11:43.726978 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769245c2-434e-4729-a415-587225cd6b61 service neutron] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:11:43.727295 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769245c2-434e-4729-a415-587225cd6b61 service neutron] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:11:43.727571 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769245c2-434e-4729-a415-587225cd6b61 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:11:43.736648 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769245c2-434e-4729-a415-587225cd6b61 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:11:43.738726 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769245c2-434e-4729-a415-587225cd6b61 service neutron] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:11:43.748192 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1294/2555] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jun 23 16:11:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 16:11:44.012135 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad1e3cb-a899-4786-bef9-ce5303269689 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:11:44.014276 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-0ad1e3cb-a899-4786-bef9-ce5303269689 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:11:44.038050 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad1e3cb-a899-4786-bef9-ce5303269689 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:11:44.043661 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de776a72-07d9-45cb-b268-cbca5fb3569a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:11:44.044547 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de776a72-07d9-45cb-b268-cbca5fb3569a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:11:44.045511 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de776a72-07d9-45cb-b268-cbca5fb3569a admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:11:44.045798 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de776a72-07d9-45cb-b268-cbca5fb3569a admin admin] PATH_INFO: `/v3/projects/4dc9b6d705724135b8c7e9579baa239c` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:11:44.047082 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de776a72-07d9-45cb-b268-cbca5fb3569a admin admin] RBAC: Authorizing `identity:delete_project(project_id=4dc9b6d705724135b8c7e9579baa239c)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:11:44.049065 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de776a72-07d9-45cb-b268-cbca5fb3569a admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:11:44.090393 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:11:44.098090 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-de776a72-07d9-45cb-b268-cbca5fb3569a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4dc9b6d705724135b8c7e9579baa239c', 'request_id': 'req-de776a72-07d9-45cb-b268-cbca5fb3569a'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:11:44.103727 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-de776a72-07d9-45cb-b268-cbca5fb3569a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4dc9b6d705724135b8c7e9579baa239c', 'request_id': 'req-de776a72-07d9-45cb-b268-cbca5fb3569a'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:11:44.111169 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-de776a72-07d9-45cb-b268-cbca5fb3569a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4dc9b6d705724135b8c7e9579baa239c', 'request_id': 'req-de776a72-07d9-45cb-b268-cbca5fb3569a'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:11:44.122184 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-de776a72-07d9-45cb-b268-cbca5fb3569a admin admin] The token cache is being invalidate because project 4dc9b6d705724135b8c7e9579baa239c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80154) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 16:11:44.122476 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-de776a72-07d9-45cb-b268-cbca5fb3569a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:11:44.123171 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-de776a72-07d9-45cb-b268-cbca5fb3569a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:11:44.124726 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1262/2556] 10.222.0.46 () {58 vars in 1232 bytes} [Thu Jun 23 16:11:44 2022] DELETE /identity/v3/projects/4dc9b6d705724135b8c7e9579baa239c => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 16:11:44.128684 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769245c2-434e-4729-a415-587225cd6b61 service neutron] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:11:44.138235 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769245c2-434e-4729-a415-587225cd6b61 service neutron] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:11:44.194736 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc593c6-1cd6-45a3-ad95-c2317dc07edf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:11:44.195530 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc593c6-1cd6-45a3-ad95-c2317dc07edf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:11:44.195815 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc593c6-1cd6-45a3-ad95-c2317dc07edf admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:11:44.196092 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc593c6-1cd6-45a3-ad95-c2317dc07edf admin admin] PATH_INFO: `/v3/users/97b7e4e0a8ac4c92844b08beb74d2844` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:11:44.197286 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc593c6-1cd6-45a3-ad95-c2317dc07edf admin admin] RBAC: Authorizing `identity:delete_user(user_id=97b7e4e0a8ac4c92844b08beb74d2844)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:11:44.199259 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc593c6-1cd6-45a3-ad95-c2317dc07edf admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:11:44.244552 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-dcc593c6-1cd6-45a3-ad95-c2317dc07edf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '97b7e4e0a8ac4c92844b08beb74d2844'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:11:44.251437 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-dcc593c6-1cd6-45a3-ad95-c2317dc07edf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '97b7e4e0a8ac4c92844b08beb74d2844'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:11:44.252100 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-dcc593c6-1cd6-45a3-ad95-c2317dc07edf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '97b7e4e0a8ac4c92844b08beb74d2844'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:11:44.252712 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-dcc593c6-1cd6-45a3-ad95-c2317dc07edf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '97b7e4e0a8ac4c92844b08beb74d2844'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:11:44.263250 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:11:44.265658 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-dcc593c6-1cd6-45a3-ad95-c2317dc07edf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '97b7e4e0a8ac4c92844b08beb74d2844'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:11:44.281030 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1295/2557] 10.222.0.46 () {58 vars in 1223 bytes} [Thu Jun 23 16:11:44 2022] DELETE /identity/v3/users/97b7e4e0a8ac4c92844b08beb74d2844 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 16:11:44.566597 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de776a72-07d9-45cb-b268-cbca5fb3569a admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:11:44.568321 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-de776a72-07d9-45cb-b268-cbca5fb3569a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:11:44.587741 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de776a72-07d9-45cb-b268-cbca5fb3569a admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:11:44.643604 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf73cd0-55ce-4bf8-b546-6746a7733179 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:11:44.645122 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf73cd0-55ce-4bf8-b546-6746a7733179 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:11:44.645405 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf73cd0-55ce-4bf8-b546-6746a7733179 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:11:44.645680 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf73cd0-55ce-4bf8-b546-6746a7733179 admin admin] PATH_INFO: `/v3/projects/0609f197e8424bdfbaa9a2e501de879b` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:11:44.646910 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf73cd0-55ce-4bf8-b546-6746a7733179 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0609f197e8424bdfbaa9a2e501de879b)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:11:44.648820 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf73cd0-55ce-4bf8-b546-6746a7733179 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:11:44.695452 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-adf73cd0-55ce-4bf8-b546-6746a7733179 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0609f197e8424bdfbaa9a2e501de879b', 'request_id': 'req-adf73cd0-55ce-4bf8-b546-6746a7733179'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:11:44.700983 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-adf73cd0-55ce-4bf8-b546-6746a7733179 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0609f197e8424bdfbaa9a2e501de879b', 'request_id': 'req-adf73cd0-55ce-4bf8-b546-6746a7733179'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:11:44.708390 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-adf73cd0-55ce-4bf8-b546-6746a7733179 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0609f197e8424bdfbaa9a2e501de879b', 'request_id': 'req-adf73cd0-55ce-4bf8-b546-6746a7733179'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:11:44.719847 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-adf73cd0-55ce-4bf8-b546-6746a7733179 admin admin] The token cache is being invalidate because project 0609f197e8424bdfbaa9a2e501de879b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80154) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 16:11:44.720173 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-adf73cd0-55ce-4bf8-b546-6746a7733179 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:11:44.720826 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-adf73cd0-55ce-4bf8-b546-6746a7733179 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:11:44.722405 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1263/2558] 10.222.0.46 () {58 vars in 1232 bytes} [Thu Jun 23 16:11:44 2022] DELETE /identity/v3/projects/0609f197e8424bdfbaa9a2e501de879b => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 16:11:44.727347 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b033746-e13d-4214-a2cb-7b94483bc5a2 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:11:44.727836 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b033746-e13d-4214-a2cb-7b94483bc5a2 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:11:44.727836 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b033746-e13d-4214-a2cb-7b94483bc5a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:11:44.746298 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-0b033746-e13d-4214-a2cb-7b94483bc5a2 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:11:44.747545 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-0b033746-e13d-4214-a2cb-7b94483bc5a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:11:44.806776 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1296/2559] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:11:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 16:11:44.814386 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce7316e-aac4-433c-b47d-1b3688caaf66 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:11:44.814683 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce7316e-aac4-433c-b47d-1b3688caaf66 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:11:44.814995 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce7316e-aac4-433c-b47d-1b3688caaf66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:11:44.835306 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-8ce7316e-aac4-433c-b47d-1b3688caaf66 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:11:44.836542 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-8ce7316e-aac4-433c-b47d-1b3688caaf66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:11:44.844984 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1264/2560] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:11:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 16:11:44.849225 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b033746-e13d-4214-a2cb-7b94483bc5a2 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:11:44.859132 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b033746-e13d-4214-a2cb-7b94483bc5a2 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:11:44.864294 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d88b1c5-45e6-40e0-9408-dd2045b50e68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:11:44.864959 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d88b1c5-45e6-40e0-9408-dd2045b50e68 admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:11:44.865248 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d88b1c5-45e6-40e0-9408-dd2045b50e68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:11:44.865525 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d88b1c5-45e6-40e0-9408-dd2045b50e68 admin admin] PATH_INFO: `/v3/domains` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:11:44.866251 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d88b1c5-45e6-40e0-9408-dd2045b50e68 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 16:11:44.866562 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d88b1c5-45e6-40e0-9408-dd2045b50e68 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:11:44.868209 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d88b1c5-45e6-40e0-9408-dd2045b50e68 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:11:44.880465 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1297/2561] 10.222.0.46 () {58 vars in 1152 bytes} [Thu Jun 23 16:11:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 16:11:44.884751 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce7316e-aac4-433c-b47d-1b3688caaf66 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:11:44.894429 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce7316e-aac4-433c-b47d-1b3688caaf66 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:11:44.899080 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1f4d4a-7ecd-4122-a248-aa523d8d3c28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:11:44.900005 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1f4d4a-7ecd-4122-a248-aa523d8d3c28 admin admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:11:44.900290 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1f4d4a-7ecd-4122-a248-aa523d8d3c28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:11:44.900567 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1f4d4a-7ecd-4122-a248-aa523d8d3c28 admin admin] PATH_INFO: `/v3/projects` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:11:44.901272 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1f4d4a-7ecd-4122-a248-aa523d8d3c28 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:11:44.903205 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1f4d4a-7ecd-4122-a248-aa523d8d3c28 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:11:44.929802 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1265/2562] 10.222.0.46 () {60 vars in 1152 bytes} [Thu Jun 23 16:11:44 2022] POST /identity/v3/projects => generated 381 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 16:11:44.933729 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d88b1c5-45e6-40e0-9408-dd2045b50e68 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:11:44.944583 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d88b1c5-45e6-40e0-9408-dd2045b50e68 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:11:45.011129 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80496649-930d-473c-b541-9868e38eaaab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:11:45.011724 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80496649-930d-473c-b541-9868e38eaaab admin admin] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:11:45.012013 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80496649-930d-473c-b541-9868e38eaaab admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:11:45.012297 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80496649-930d-473c-b541-9868e38eaaab admin admin] PATH_INFO: `/v3/users` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:11:45.013022 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80496649-930d-473c-b541-9868e38eaaab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:11:45.015035 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80496649-930d-473c-b541-9868e38eaaab admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:11:45.035897 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1298/2563] 10.222.0.46 () {60 vars in 1143 bytes} [Thu Jun 23 16:11:44 2022] POST /identity/v3/users => generated 291 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 16:11:45.040053 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1f4d4a-7ecd-4122-a248-aa523d8d3c28 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:11:45.051656 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1f4d4a-7ecd-4122-a248-aa523d8d3c28 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:11:45.057129 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5dbf00-457a-4e34-b55a-1ee2ce2cd9de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:11:45.057505 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5dbf00-457a-4e34-b55a-1ee2ce2cd9de admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:11:45.057792 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5dbf00-457a-4e34-b55a-1ee2ce2cd9de admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:11:45.058075 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5dbf00-457a-4e34-b55a-1ee2ce2cd9de admin admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:11:45.058848 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5dbf00-457a-4e34-b55a-1ee2ce2cd9de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:11:45.060455 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5dbf00-457a-4e34-b55a-1ee2ce2cd9de admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:11:45.072442 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1266/2564] 10.222.0.46 () {58 vars in 1121 bytes} [Thu Jun 23 16:11:45 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 16:11:45.075324 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80496649-930d-473c-b541-9868e38eaaab admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:11:45.085525 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80496649-930d-473c-b541-9868e38eaaab admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:11:45.090255 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf02a09-7a90-4a03-9d11-a3d9be675cd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:11:45.090961 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf02a09-7a90-4a03-9d11-a3d9be675cd7 admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:11:45.091246 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf02a09-7a90-4a03-9d11-a3d9be675cd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:11:45.091526 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf02a09-7a90-4a03-9d11-a3d9be675cd7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:11:45.092273 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf02a09-7a90-4a03-9d11-a3d9be675cd7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:11:45.093879 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf02a09-7a90-4a03-9d11-a3d9be675cd7 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:11:45.105006 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1299/2565] 10.222.0.46 () {58 vars in 1121 bytes} [Thu Jun 23 16:11:45 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 16:11:45.109053 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5dbf00-457a-4e34-b55a-1ee2ce2cd9de admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:11:45.119865 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5dbf00-457a-4e34-b55a-1ee2ce2cd9de admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:11:45.124460 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d60ccd7-cb50-4f7b-9481-077f3382f61d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:11:45.125349 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d60ccd7-cb50-4f7b-9481-077f3382f61d admin admin] REQUEST_METHOD: `PUT` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:11:45.125633 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d60ccd7-cb50-4f7b-9481-077f3382f61d admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:11:45.125912 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d60ccd7-cb50-4f7b-9481-077f3382f61d admin admin] PATH_INFO: `/v3/projects/8a3e6b11fd5445cdb3ed902b05f72857/users/1f232ee589494ba499dcee3182e5dfab/roles/e2a5bbf3200a4d4091d72bc7ff78c34d` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:11:45.140203 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d60ccd7-cb50-4f7b-9481-077f3382f61d admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a3e6b11fd5445cdb3ed902b05f72857, user_id=1f232ee589494ba499dcee3182e5dfab, role_id=e2a5bbf3200a4d4091d72bc7ff78c34d)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:11:45.142884 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d60ccd7-cb50-4f7b-9481-077f3382f61d admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:11:45.145896 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=85,keystone:DELETE=11,keystone:INSERT=4 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:11:45.158846 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1267/2566] 10.222.0.46 () {60 vars in 1482 bytes} [Thu Jun 23 16:11:45 2022] PUT /identity/v3/projects/8a3e6b11fd5445cdb3ed902b05f72857/users/1f232ee589494ba499dcee3182e5dfab/roles/e2a5bbf3200a4d4091d72bc7ff78c34d => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 16:11:45.169326 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b29fab9-520f-4ba9-b0b7-c3cd490c7293 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:11:45.169622 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b29fab9-520f-4ba9-b0b7-c3cd490c7293 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:11:45.169907 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b29fab9-520f-4ba9-b0b7-c3cd490c7293 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:11:45.197814 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-2b29fab9-520f-4ba9-b0b7-c3cd490c7293 None None] MFA Rules not processed for user `1f232ee589494ba499dcee3182e5dfab`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:11:45.199181 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-2b29fab9-520f-4ba9-b0b7-c3cd490c7293 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:11:45.240418 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:DELETE=2,keystone:SELECT=94,keystone:INSERT=4 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:11:45.263915 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1300/2567] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:11:45 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 16:11:45.268272 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d60ccd7-cb50-4f7b-9481-077f3382f61d admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:11:45.330551 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d60ccd7-cb50-4f7b-9481-077f3382f61d admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:11:45.335193 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68119651-6a3f-42aa-be87-c34607c8d0ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:11:45.335847 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68119651-6a3f-42aa-be87-c34607c8d0ef admin admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:11:45.336130 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68119651-6a3f-42aa-be87-c34607c8d0ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:11:45.336624 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68119651-6a3f-42aa-be87-c34607c8d0ef admin admin] PATH_INFO: `/v3/projects` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:11:45.337088 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68119651-6a3f-42aa-be87-c34607c8d0ef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:11:45.339047 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68119651-6a3f-42aa-be87-c34607c8d0ef admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:11:45.365089 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1268/2568] 10.222.0.46 () {60 vars in 1152 bytes} [Thu Jun 23 16:11:45 2022] POST /identity/v3/projects => generated 381 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 16:11:45.369421 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b29fab9-520f-4ba9-b0b7-c3cd490c7293 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:11:45.380234 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b29fab9-520f-4ba9-b0b7-c3cd490c7293 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:11:45.436832 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1d6f11-cd0b-4dbc-912d-ece8157220a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:11:45.437506 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1d6f11-cd0b-4dbc-912d-ece8157220a8 admin admin] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:11:45.437793 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1d6f11-cd0b-4dbc-912d-ece8157220a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:11:45.438075 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1d6f11-cd0b-4dbc-912d-ece8157220a8 admin admin] PATH_INFO: `/v3/users` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:11:45.438808 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1d6f11-cd0b-4dbc-912d-ece8157220a8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:11:45.440816 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1d6f11-cd0b-4dbc-912d-ece8157220a8 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:11:45.460662 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1301/2569] 10.222.0.46 () {60 vars in 1143 bytes} [Thu Jun 23 16:11:45 2022] POST /identity/v3/users => generated 297 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 16:11:45.464797 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68119651-6a3f-42aa-be87-c34607c8d0ef admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:11:45.476008 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68119651-6a3f-42aa-be87-c34607c8d0ef admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:11:45.480648 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09320633-e9f1-47a3-b648-391c2f6243ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:11:45.481313 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09320633-e9f1-47a3-b648-391c2f6243ed admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:11:45.481598 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09320633-e9f1-47a3-b648-391c2f6243ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:11:45.481871 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09320633-e9f1-47a3-b648-391c2f6243ed admin admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:11:45.482595 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09320633-e9f1-47a3-b648-391c2f6243ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:11:45.484216 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09320633-e9f1-47a3-b648-391c2f6243ed admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:11:45.495607 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1269/2570] 10.222.0.46 () {58 vars in 1121 bytes} [Thu Jun 23 16:11:45 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 16:11:45.499722 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1d6f11-cd0b-4dbc-912d-ece8157220a8 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:11:45.510290 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1d6f11-cd0b-4dbc-912d-ece8157220a8 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:11:45.515040 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d233ebe-5489-4dd4-a758-d1937f5b5555 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:11:45.515899 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d233ebe-5489-4dd4-a758-d1937f5b5555 admin admin] REQUEST_METHOD: `PUT` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:11:45.516186 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d233ebe-5489-4dd4-a758-d1937f5b5555 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:11:45.516467 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d233ebe-5489-4dd4-a758-d1937f5b5555 admin admin] PATH_INFO: `/v3/projects/ef6eb19ef87947a6b57788eadf647cdc/users/7f8ef9a677ee40269bbaa22d5e8a2a76/roles/812d3c567c52469cb2b35709fbdc857c` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:11:45.531431 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d233ebe-5489-4dd4-a758-d1937f5b5555 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef6eb19ef87947a6b57788eadf647cdc, user_id=7f8ef9a677ee40269bbaa22d5e8a2a76, role_id=812d3c567c52469cb2b35709fbdc857c)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:11:45.534129 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d233ebe-5489-4dd4-a758-d1937f5b5555 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:11:45.547221 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1302/2571] 10.222.0.46 () {60 vars in 1482 bytes} [Thu Jun 23 16:11:45 2022] PUT /identity/v3/projects/ef6eb19ef87947a6b57788eadf647cdc/users/7f8ef9a677ee40269bbaa22d5e8a2a76/roles/812d3c567c52469cb2b35709fbdc857c => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 16:11:45.551667 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09320633-e9f1-47a3-b648-391c2f6243ed admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:11:45.561994 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09320633-e9f1-47a3-b648-391c2f6243ed admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:11:45.618729 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7091f9-a8a8-43f8-8640-984fdc6e20aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:11:45.619396 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7091f9-a8a8-43f8-8640-984fdc6e20aa admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:11:45.619683 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7091f9-a8a8-43f8-8640-984fdc6e20aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:11:45.619968 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7091f9-a8a8-43f8-8640-984fdc6e20aa admin admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:11:45.620731 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7091f9-a8a8-43f8-8640-984fdc6e20aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:11:45.622350 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7091f9-a8a8-43f8-8640-984fdc6e20aa admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:11:45.632641 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1270/2572] 10.222.0.46 () {58 vars in 1121 bytes} [Thu Jun 23 16:11:45 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 16:11:45.636971 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d233ebe-5489-4dd4-a758-d1937f5b5555 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:11:45.647859 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d233ebe-5489-4dd4-a758-d1937f5b5555 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:11:45.652792 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb3af90-ec5b-4d69-8481-04f4008b8212 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:11:45.653701 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb3af90-ec5b-4d69-8481-04f4008b8212 admin admin] REQUEST_METHOD: `PUT` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:11:45.654020 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb3af90-ec5b-4d69-8481-04f4008b8212 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:11:45.654306 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb3af90-ec5b-4d69-8481-04f4008b8212 admin admin] PATH_INFO: `/v3/projects/ef6eb19ef87947a6b57788eadf647cdc/users/7f8ef9a677ee40269bbaa22d5e8a2a76/roles/812d3c567c52469cb2b35709fbdc857c` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:11:45.656397 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb3af90-ec5b-4d69-8481-04f4008b8212 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef6eb19ef87947a6b57788eadf647cdc, user_id=7f8ef9a677ee40269bbaa22d5e8a2a76, role_id=812d3c567c52469cb2b35709fbdc857c)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:11:45.659095 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb3af90-ec5b-4d69-8481-04f4008b8212 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:11:45.669937 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1303/2573] 10.222.0.46 () {60 vars in 1482 bytes} [Thu Jun 23 16:11:45 2022] PUT /identity/v3/projects/ef6eb19ef87947a6b57788eadf647cdc/users/7f8ef9a677ee40269bbaa22d5e8a2a76/roles/812d3c567c52469cb2b35709fbdc857c => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 16:11:45.681871 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f34fece-2f9e-4f53-9cc8-641a305b051b None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:11:45.682169 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f34fece-2f9e-4f53-9cc8-641a305b051b None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:11:45.682459 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f34fece-2f9e-4f53-9cc8-641a305b051b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:11:45.709175 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-9f34fece-2f9e-4f53-9cc8-641a305b051b None None] MFA Rules not processed for user `7f8ef9a677ee40269bbaa22d5e8a2a76`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:11:45.710406 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-9f34fece-2f9e-4f53-9cc8-641a305b051b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:11:45.760668 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:11:45.778841 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1271/2574] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:11:45 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 16:11:45.799432 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb3af90-ec5b-4d69-8481-04f4008b8212 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:11:45.801699 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-2bb3af90-ec5b-4d69-8481-04f4008b8212 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:11:45.859295 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb3af90-ec5b-4d69-8481-04f4008b8212 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:11:45.863144 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20848f2-f5b9-482a-b5b8-eff9798055e4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce18eaaba3a6477684fff1fd98ec8b51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:11:45.864309 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20848f2-f5b9-482a-b5b8-eff9798055e4 service cinder] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:11:45.864597 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20848f2-f5b9-482a-b5b8-eff9798055e4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:11:45.864882 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20848f2-f5b9-482a-b5b8-eff9798055e4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:11:45.874305 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20848f2-f5b9-482a-b5b8-eff9798055e4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:11:45.875843 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20848f2-f5b9-482a-b5b8-eff9798055e4 service cinder] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:11:45.885502 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1304/2575] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jun 23 16:11:45 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 16:11:46.188625 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f34fece-2f9e-4f53-9cc8-641a305b051b None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:11:46.197777 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f34fece-2f9e-4f53-9cc8-641a305b051b None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:11:46.202427 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e022fd-cf7b-4252-b39f-a573b3d6f498 tempest-VolumeQuotaClassesTest-1933692156 tempest-VolumeQuotaClassesTest-1933692156-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f8ef9a677ee40269bbaa22d5e8a2a76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef6eb19ef87947a6b57788eadf647cdc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:11:46.203106 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e022fd-cf7b-4252-b39f-a573b3d6f498 tempest-VolumeQuotaClassesTest-1933692156 tempest-VolumeQuotaClassesTest-1933692156-project-admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:11:46.203399 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e022fd-cf7b-4252-b39f-a573b3d6f498 tempest-VolumeQuotaClassesTest-1933692156 tempest-VolumeQuotaClassesTest-1933692156-project-admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:11:46.203688 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e022fd-cf7b-4252-b39f-a573b3d6f498 tempest-VolumeQuotaClassesTest-1933692156 tempest-VolumeQuotaClassesTest-1933692156-project-admin] PATH_INFO: `/v3/domains` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:11:46.204418 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e022fd-cf7b-4252-b39f-a573b3d6f498 tempest-VolumeQuotaClassesTest-1933692156 tempest-VolumeQuotaClassesTest-1933692156-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 16:11:46.204731 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e022fd-cf7b-4252-b39f-a573b3d6f498 tempest-VolumeQuotaClassesTest-1933692156 tempest-VolumeQuotaClassesTest-1933692156-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:11:46.206333 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e022fd-cf7b-4252-b39f-a573b3d6f498 tempest-VolumeQuotaClassesTest-1933692156 tempest-VolumeQuotaClassesTest-1933692156-project-admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:11:46.218630 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1272/2576] 10.222.0.46 () {58 vars in 1152 bytes} [Thu Jun 23 16:11:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 16:11:46.223546 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20848f2-f5b9-482a-b5b8-eff9798055e4 service cinder] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:11:46.232778 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20848f2-f5b9-482a-b5b8-eff9798055e4 service cinder] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:11:46.237392 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddb1ae6-d896-4210-b586-6070c9f21fb5 tempest-VolumeQuotaClassesTest-1933692156 tempest-VolumeQuotaClassesTest-1933692156-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f8ef9a677ee40269bbaa22d5e8a2a76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef6eb19ef87947a6b57788eadf647cdc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:11:46.238060 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddb1ae6-d896-4210-b586-6070c9f21fb5 tempest-VolumeQuotaClassesTest-1933692156 tempest-VolumeQuotaClassesTest-1933692156-project-admin] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:11:46.238353 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddb1ae6-d896-4210-b586-6070c9f21fb5 tempest-VolumeQuotaClassesTest-1933692156 tempest-VolumeQuotaClassesTest-1933692156-project-admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:11:46.238637 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddb1ae6-d896-4210-b586-6070c9f21fb5 tempest-VolumeQuotaClassesTest-1933692156 tempest-VolumeQuotaClassesTest-1933692156-project-admin] PATH_INFO: `/v3/projects` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:11:46.239403 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ddb1ae6-d896-4210-b586-6070c9f21fb5 tempest-VolumeQuotaClassesTest-1933692156 tempest-VolumeQuotaClassesTest-1933692156-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:11:46.241321 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ddb1ae6-d896-4210-b586-6070c9f21fb5 tempest-VolumeQuotaClassesTest-1933692156 tempest-VolumeQuotaClassesTest-1933692156-project-admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:11:46.268493 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1305/2577] 10.222.0.46 () {60 vars in 1152 bytes} [Thu Jun 23 16:11:46 2022] POST /identity/v3/projects => generated 354 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 16:11:46.273490 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e022fd-cf7b-4252-b39f-a573b3d6f498 tempest-VolumeQuotaClassesTest-1933692156 tempest-VolumeQuotaClassesTest-1933692156-project-admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:11:46.283678 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e022fd-cf7b-4252-b39f-a573b3d6f498 tempest-VolumeQuotaClassesTest-1933692156 tempest-VolumeQuotaClassesTest-1933692156-project-admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:11:46.339394 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed402e37-9e6a-42be-bfcc-42eb0b4e5b36 tempest-VolumeQuotaClassesTest-1933692156 tempest-VolumeQuotaClassesTest-1933692156-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f8ef9a677ee40269bbaa22d5e8a2a76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef6eb19ef87947a6b57788eadf647cdc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:11:46.340074 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed402e37-9e6a-42be-bfcc-42eb0b4e5b36 tempest-VolumeQuotaClassesTest-1933692156 tempest-VolumeQuotaClassesTest-1933692156-project-admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:11:46.340367 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed402e37-9e6a-42be-bfcc-42eb0b4e5b36 tempest-VolumeQuotaClassesTest-1933692156 tempest-VolumeQuotaClassesTest-1933692156-project-admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:11:46.340654 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed402e37-9e6a-42be-bfcc-42eb0b4e5b36 tempest-VolumeQuotaClassesTest-1933692156 tempest-VolumeQuotaClassesTest-1933692156-project-admin] PATH_INFO: `/v3/domains` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:11:46.341380 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed402e37-9e6a-42be-bfcc-42eb0b4e5b36 tempest-VolumeQuotaClassesTest-1933692156 tempest-VolumeQuotaClassesTest-1933692156-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 16:11:46.341697 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed402e37-9e6a-42be-bfcc-42eb0b4e5b36 tempest-VolumeQuotaClassesTest-1933692156 tempest-VolumeQuotaClassesTest-1933692156-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:11:46.343349 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed402e37-9e6a-42be-bfcc-42eb0b4e5b36 tempest-VolumeQuotaClassesTest-1933692156 tempest-VolumeQuotaClassesTest-1933692156-project-admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:11:46.355430 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1273/2578] 10.222.0.46 () {58 vars in 1152 bytes} [Thu Jun 23 16:11:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 23 16:11:46.391579 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddb1ae6-d896-4210-b586-6070c9f21fb5 tempest-VolumeQuotaClassesTest-1933692156 tempest-VolumeQuotaClassesTest-1933692156-project-admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:11:46.402653 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddb1ae6-d896-4210-b586-6070c9f21fb5 tempest-VolumeQuotaClassesTest-1933692156 tempest-VolumeQuotaClassesTest-1933692156-project-admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:11:46.407260 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a19730d-d0c7-48bb-8a2c-2015a4bfc2fe tempest-VolumeQuotaClassesTest-1933692156 tempest-VolumeQuotaClassesTest-1933692156-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f8ef9a677ee40269bbaa22d5e8a2a76', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef6eb19ef87947a6b57788eadf647cdc', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:11:46.408142 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a19730d-d0c7-48bb-8a2c-2015a4bfc2fe tempest-VolumeQuotaClassesTest-1933692156 tempest-VolumeQuotaClassesTest-1933692156-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:11:46.408435 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a19730d-d0c7-48bb-8a2c-2015a4bfc2fe tempest-VolumeQuotaClassesTest-1933692156 tempest-VolumeQuotaClassesTest-1933692156-project-admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:11:46.408715 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a19730d-d0c7-48bb-8a2c-2015a4bfc2fe tempest-VolumeQuotaClassesTest-1933692156 tempest-VolumeQuotaClassesTest-1933692156-project-admin] PATH_INFO: `/v3/projects/0d8509f179c84b38ae081e51b246ca80` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:11:46.409883 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a19730d-d0c7-48bb-8a2c-2015a4bfc2fe tempest-VolumeQuotaClassesTest-1933692156 tempest-VolumeQuotaClassesTest-1933692156-project-admin] RBAC: Authorizing `identity:delete_project(project_id=0d8509f179c84b38ae081e51b246ca80)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:11:46.412383 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a19730d-d0c7-48bb-8a2c-2015a4bfc2fe tempest-VolumeQuotaClassesTest-1933692156 tempest-VolumeQuotaClassesTest-1933692156-project-admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:11:46.423746 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:11:46.461104 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-9a19730d-d0c7-48bb-8a2c-2015a4bfc2fe tempest-VolumeQuotaClassesTest-1933692156 tempest-VolumeQuotaClassesTest-1933692156-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0d8509f179c84b38ae081e51b246ca80', 'request_id': 'req-9a19730d-d0c7-48bb-8a2c-2015a4bfc2fe'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:11:46.466948 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-9a19730d-d0c7-48bb-8a2c-2015a4bfc2fe tempest-VolumeQuotaClassesTest-1933692156 tempest-VolumeQuotaClassesTest-1933692156-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0d8509f179c84b38ae081e51b246ca80', 'request_id': 'req-9a19730d-d0c7-48bb-8a2c-2015a4bfc2fe'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:11:46.474168 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-9a19730d-d0c7-48bb-8a2c-2015a4bfc2fe tempest-VolumeQuotaClassesTest-1933692156 tempest-VolumeQuotaClassesTest-1933692156-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0d8509f179c84b38ae081e51b246ca80', 'request_id': 'req-9a19730d-d0c7-48bb-8a2c-2015a4bfc2fe'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:11:46.486781 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-9a19730d-d0c7-48bb-8a2c-2015a4bfc2fe tempest-VolumeQuotaClassesTest-1933692156 tempest-VolumeQuotaClassesTest-1933692156-project-admin] The token cache is being invalidate because project 0d8509f179c84b38ae081e51b246ca80 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80153) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 16:11:46.487154 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-9a19730d-d0c7-48bb-8a2c-2015a4bfc2fe tempest-VolumeQuotaClassesTest-1933692156 tempest-VolumeQuotaClassesTest-1933692156-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:11:46.487807 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-9a19730d-d0c7-48bb-8a2c-2015a4bfc2fe tempest-VolumeQuotaClassesTest-1933692156 tempest-VolumeQuotaClassesTest-1933692156-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:11:46.489354 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1306/2579] 10.222.0.46 () {58 vars in 1232 bytes} [Thu Jun 23 16:11:46 2022] DELETE /identity/v3/projects/0d8509f179c84b38ae081e51b246ca80 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 16:11:46.685492 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed402e37-9e6a-42be-bfcc-42eb0b4e5b36 tempest-VolumeQuotaClassesTest-1933692156 tempest-VolumeQuotaClassesTest-1933692156-project-admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:11:46.687403 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-ed402e37-9e6a-42be-bfcc-42eb0b4e5b36 tempest-VolumeQuotaClassesTest-1933692156 tempest-VolumeQuotaClassesTest-1933692156-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:11:46.754982 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed402e37-9e6a-42be-bfcc-42eb0b4e5b36 tempest-VolumeQuotaClassesTest-1933692156 tempest-VolumeQuotaClassesTest-1933692156-project-admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:11:46.811344 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb60bbc-eb82-4e4c-b97b-d683db644c25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:11:46.812298 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb60bbc-eb82-4e4c-b97b-d683db644c25 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:11:46.812687 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb60bbc-eb82-4e4c-b97b-d683db644c25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:11:46.813022 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb60bbc-eb82-4e4c-b97b-d683db644c25 admin admin] PATH_INFO: `/v3/users/1f232ee589494ba499dcee3182e5dfab` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:11:46.814291 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb60bbc-eb82-4e4c-b97b-d683db644c25 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1f232ee589494ba499dcee3182e5dfab)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:11:46.816818 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb60bbc-eb82-4e4c-b97b-d683db644c25 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:11:46.839737 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:11:46.865734 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-aeb60bbc-eb82-4e4c-b97b-d683db644c25 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1f232ee589494ba499dcee3182e5dfab'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:11:46.866405 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-aeb60bbc-eb82-4e4c-b97b-d683db644c25 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1f232ee589494ba499dcee3182e5dfab'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:11:46.874091 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-aeb60bbc-eb82-4e4c-b97b-d683db644c25 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1f232ee589494ba499dcee3182e5dfab'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:11:46.881745 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-aeb60bbc-eb82-4e4c-b97b-d683db644c25 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1f232ee589494ba499dcee3182e5dfab'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:11:46.882551 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-aeb60bbc-eb82-4e4c-b97b-d683db644c25 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1f232ee589494ba499dcee3182e5dfab'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:11:46.900848 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1274/2580] 10.222.0.46 () {58 vars in 1223 bytes} [Thu Jun 23 16:11:46 2022] DELETE /identity/v3/users/1f232ee589494ba499dcee3182e5dfab => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 16:11:46.905610 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b18a8fa-9890-4f4a-95ab-a1390a217634 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:11:46.906100 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b18a8fa-9890-4f4a-95ab-a1390a217634 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:11:46.906213 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b18a8fa-9890-4f4a-95ab-a1390a217634 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:11:46.923656 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-5b18a8fa-9890-4f4a-95ab-a1390a217634 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:11:46.924866 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-5b18a8fa-9890-4f4a-95ab-a1390a217634 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:11:46.983294 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1307/2581] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:11:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 16:11:46.998009 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb60bbc-eb82-4e4c-b97b-d683db644c25 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:11:46.999462 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-aeb60bbc-eb82-4e4c-b97b-d683db644c25 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:11:47.059641 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb60bbc-eb82-4e4c-b97b-d683db644c25 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:11:47.063372 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd11777b-14e3-4d3b-a429-1c7aacf6040a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51784064951a4619ae09b9167fb0fd0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:11:47.064032 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd11777b-14e3-4d3b-a429-1c7aacf6040a service neutron] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:11:47.064314 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd11777b-14e3-4d3b-a429-1c7aacf6040a service neutron] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:11:47.064594 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd11777b-14e3-4d3b-a429-1c7aacf6040a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:11:47.075765 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd11777b-14e3-4d3b-a429-1c7aacf6040a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:11:47.077214 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd11777b-14e3-4d3b-a429-1c7aacf6040a service neutron] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:11:47.087518 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1275/2582] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jun 23 16:11:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 16:11:47.364103 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b18a8fa-9890-4f4a-95ab-a1390a217634 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:11:47.365705 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-5b18a8fa-9890-4f4a-95ab-a1390a217634 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:11:47.381874 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b18a8fa-9890-4f4a-95ab-a1390a217634 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:11:47.387307 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8c7a22-34e3-40c9-9638-f86ec6e2e24f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:11:47.388274 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8c7a22-34e3-40c9-9638-f86ec6e2e24f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:11:47.388459 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8c7a22-34e3-40c9-9638-f86ec6e2e24f admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:11:47.388789 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8c7a22-34e3-40c9-9638-f86ec6e2e24f admin admin] PATH_INFO: `/v3/projects/8a3e6b11fd5445cdb3ed902b05f72857` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:11:47.389914 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8c7a22-34e3-40c9-9638-f86ec6e2e24f admin admin] RBAC: Authorizing `identity:delete_project(project_id=8a3e6b11fd5445cdb3ed902b05f72857)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:11:47.391880 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8c7a22-34e3-40c9-9638-f86ec6e2e24f admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:11:47.440051 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-7e8c7a22-34e3-40c9-9638-f86ec6e2e24f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8a3e6b11fd5445cdb3ed902b05f72857', 'request_id': 'req-7e8c7a22-34e3-40c9-9638-f86ec6e2e24f'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:11:47.445425 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-7e8c7a22-34e3-40c9-9638-f86ec6e2e24f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8a3e6b11fd5445cdb3ed902b05f72857', 'request_id': 'req-7e8c7a22-34e3-40c9-9638-f86ec6e2e24f'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:11:47.451590 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-7e8c7a22-34e3-40c9-9638-f86ec6e2e24f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8a3e6b11fd5445cdb3ed902b05f72857', 'request_id': 'req-7e8c7a22-34e3-40c9-9638-f86ec6e2e24f'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:11:47.464195 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-7e8c7a22-34e3-40c9-9638-f86ec6e2e24f admin admin] The token cache is being invalidate because project 8a3e6b11fd5445cdb3ed902b05f72857 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80153) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 16:11:47.464486 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-7e8c7a22-34e3-40c9-9638-f86ec6e2e24f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:11:47.465112 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-7e8c7a22-34e3-40c9-9638-f86ec6e2e24f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:11:47.466647 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1308/2583] 10.222.0.46 () {58 vars in 1232 bytes} [Thu Jun 23 16:11:47 2022] DELETE /identity/v3/projects/8a3e6b11fd5445cdb3ed902b05f72857 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 16:11:47.470614 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd11777b-14e3-4d3b-a429-1c7aacf6040a service neutron] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:11:47.480909 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd11777b-14e3-4d3b-a429-1c7aacf6040a service neutron] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:11:47.540257 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc650e8-085f-432f-b7c3-612080a0df91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:11:47.541159 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc650e8-085f-432f-b7c3-612080a0df91 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:11:47.541449 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc650e8-085f-432f-b7c3-612080a0df91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:11:47.541729 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc650e8-085f-432f-b7c3-612080a0df91 admin admin] PATH_INFO: `/v3/users/7f8ef9a677ee40269bbaa22d5e8a2a76` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:11:47.543034 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc650e8-085f-432f-b7c3-612080a0df91 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7f8ef9a677ee40269bbaa22d5e8a2a76)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:11:47.545032 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc650e8-085f-432f-b7c3-612080a0df91 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:11:47.601492 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:11:47.605189 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-4dc650e8-085f-432f-b7c3-612080a0df91 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7f8ef9a677ee40269bbaa22d5e8a2a76'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:11:47.605975 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-4dc650e8-085f-432f-b7c3-612080a0df91 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7f8ef9a677ee40269bbaa22d5e8a2a76'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:11:47.616724 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-4dc650e8-085f-432f-b7c3-612080a0df91 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7f8ef9a677ee40269bbaa22d5e8a2a76'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:11:47.624419 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-4dc650e8-085f-432f-b7c3-612080a0df91 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7f8ef9a677ee40269bbaa22d5e8a2a76'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:11:47.625116 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-4dc650e8-085f-432f-b7c3-612080a0df91 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7f8ef9a677ee40269bbaa22d5e8a2a76'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:11:47.644279 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1276/2584] 10.222.0.46 () {58 vars in 1223 bytes} [Thu Jun 23 16:11:47 2022] DELETE /identity/v3/users/7f8ef9a677ee40269bbaa22d5e8a2a76 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 16:11:47.919727 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8c7a22-34e3-40c9-9638-f86ec6e2e24f admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:11:47.921318 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-7e8c7a22-34e3-40c9-9638-f86ec6e2e24f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:11:47.936539 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8c7a22-34e3-40c9-9638-f86ec6e2e24f admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:11:47.956133 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:11:48.002054 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfaab24f-c357-4162-bba5-28d7ae3a16a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:11:48.002890 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfaab24f-c357-4162-bba5-28d7ae3a16a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:11:48.003169 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfaab24f-c357-4162-bba5-28d7ae3a16a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:11:48.003438 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfaab24f-c357-4162-bba5-28d7ae3a16a1 admin admin] PATH_INFO: `/v3/projects/ef6eb19ef87947a6b57788eadf647cdc` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:11:48.005158 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfaab24f-c357-4162-bba5-28d7ae3a16a1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ef6eb19ef87947a6b57788eadf647cdc)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:11:48.007160 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfaab24f-c357-4162-bba5-28d7ae3a16a1 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:11:48.058503 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-bfaab24f-c357-4162-bba5-28d7ae3a16a1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ef6eb19ef87947a6b57788eadf647cdc', 'request_id': 'req-bfaab24f-c357-4162-bba5-28d7ae3a16a1'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:11:48.064592 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-bfaab24f-c357-4162-bba5-28d7ae3a16a1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ef6eb19ef87947a6b57788eadf647cdc', 'request_id': 'req-bfaab24f-c357-4162-bba5-28d7ae3a16a1'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:11:48.073012 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-bfaab24f-c357-4162-bba5-28d7ae3a16a1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ef6eb19ef87947a6b57788eadf647cdc', 'request_id': 'req-bfaab24f-c357-4162-bba5-28d7ae3a16a1'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:11:48.086662 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-bfaab24f-c357-4162-bba5-28d7ae3a16a1 admin admin] The token cache is being invalidate because project ef6eb19ef87947a6b57788eadf647cdc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80153) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 16:11:48.086992 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-bfaab24f-c357-4162-bba5-28d7ae3a16a1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:11:48.087620 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-bfaab24f-c357-4162-bba5-28d7ae3a16a1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:11:48.090737 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1309/2585] 10.222.0.46 () {58 vars in 1232 bytes} [Thu Jun 23 16:11:47 2022] DELETE /identity/v3/projects/ef6eb19ef87947a6b57788eadf647cdc => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 16:11:48.094091 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfd7f18-ae8f-4b7d-a50d-46213e551229 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:11:48.094981 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfd7f18-ae8f-4b7d-a50d-46213e551229 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:11:48.095268 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfd7f18-ae8f-4b7d-a50d-46213e551229 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:11:48.115078 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-9cfd7f18-ae8f-4b7d-a50d-46213e551229 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:11:48.116347 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-9cfd7f18-ae8f-4b7d-a50d-46213e551229 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:11:48.175986 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1277/2586] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:11:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 16:11:48.183873 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc06fcb-f589-4bf7-a905-3ad975b8ae8b None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:11:48.184169 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc06fcb-f589-4bf7-a905-3ad975b8ae8b None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:11:48.184477 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc06fcb-f589-4bf7-a905-3ad975b8ae8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:11:48.203555 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-6bc06fcb-f589-4bf7-a905-3ad975b8ae8b None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:11:48.204773 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-6bc06fcb-f589-4bf7-a905-3ad975b8ae8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:11:48.214085 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1310/2587] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:11:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 16:11:48.218340 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfd7f18-ae8f-4b7d-a50d-46213e551229 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:11:48.228142 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfd7f18-ae8f-4b7d-a50d-46213e551229 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:11:48.232770 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7736d222-0a53-4b8a-aa00-f0b63c9a36bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:11:48.233435 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7736d222-0a53-4b8a-aa00-f0b63c9a36bf admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:11:48.233726 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7736d222-0a53-4b8a-aa00-f0b63c9a36bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:11:48.234299 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7736d222-0a53-4b8a-aa00-f0b63c9a36bf admin admin] PATH_INFO: `/v3/domains` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:11:48.234761 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7736d222-0a53-4b8a-aa00-f0b63c9a36bf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 16:11:48.235075 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7736d222-0a53-4b8a-aa00-f0b63c9a36bf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:11:48.236673 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7736d222-0a53-4b8a-aa00-f0b63c9a36bf admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:11:48.249184 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1278/2588] 10.222.0.46 () {58 vars in 1152 bytes} [Thu Jun 23 16:11:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 16:11:48.254471 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc06fcb-f589-4bf7-a905-3ad975b8ae8b None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:11:48.263742 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc06fcb-f589-4bf7-a905-3ad975b8ae8b None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:11:48.268585 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ce0796-6ffa-45f7-b20a-beaae0855d2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:11:48.269240 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ce0796-6ffa-45f7-b20a-beaae0855d2a admin admin] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:11:48.270227 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ce0796-6ffa-45f7-b20a-beaae0855d2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:11:48.270509 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ce0796-6ffa-45f7-b20a-beaae0855d2a admin admin] PATH_INFO: `/v3/projects` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:11:48.271383 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ce0796-6ffa-45f7-b20a-beaae0855d2a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:11:48.273283 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ce0796-6ffa-45f7-b20a-beaae0855d2a admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:11:48.300525 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1311/2589] 10.222.0.46 () {60 vars in 1152 bytes} [Thu Jun 23 16:11:48 2022] POST /identity/v3/projects => generated 383 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 16:11:48.304763 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7736d222-0a53-4b8a-aa00-f0b63c9a36bf admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:11:48.314947 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7736d222-0a53-4b8a-aa00-f0b63c9a36bf admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:11:48.375359 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d5b98d-69ae-43d6-9244-3f4d517c38f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:11:48.376196 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d5b98d-69ae-43d6-9244-3f4d517c38f9 admin admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:11:48.376486 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d5b98d-69ae-43d6-9244-3f4d517c38f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:11:48.376759 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d5b98d-69ae-43d6-9244-3f4d517c38f9 admin admin] PATH_INFO: `/v3/users` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:11:48.377477 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d5b98d-69ae-43d6-9244-3f4d517c38f9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:11:48.379991 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d5b98d-69ae-43d6-9244-3f4d517c38f9 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:11:48.401647 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1279/2590] 10.222.0.46 () {60 vars in 1143 bytes} [Thu Jun 23 16:11:48 2022] POST /identity/v3/users => generated 292 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 16:11:48.405628 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ce0796-6ffa-45f7-b20a-beaae0855d2a admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:11:48.417113 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ce0796-6ffa-45f7-b20a-beaae0855d2a admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:11:48.422829 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75e5a1f-e2d2-4795-9802-6edd092395af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:11:48.423578 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75e5a1f-e2d2-4795-9802-6edd092395af admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:11:48.423928 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75e5a1f-e2d2-4795-9802-6edd092395af admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:11:48.424287 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75e5a1f-e2d2-4795-9802-6edd092395af admin admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:11:48.425071 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75e5a1f-e2d2-4795-9802-6edd092395af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:11:48.427279 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75e5a1f-e2d2-4795-9802-6edd092395af admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:11:48.440244 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1312/2591] 10.222.0.46 () {58 vars in 1121 bytes} [Thu Jun 23 16:11:48 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 16:11:48.446888 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d5b98d-69ae-43d6-9244-3f4d517c38f9 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:11:48.455472 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d5b98d-69ae-43d6-9244-3f4d517c38f9 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:11:48.461149 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b77ed77-0fc7-4664-bf74-09aeca9922d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:11:48.461829 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b77ed77-0fc7-4664-bf74-09aeca9922d0 admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:11:48.462278 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b77ed77-0fc7-4664-bf74-09aeca9922d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:11:48.462562 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b77ed77-0fc7-4664-bf74-09aeca9922d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:11:48.463368 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b77ed77-0fc7-4664-bf74-09aeca9922d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:11:48.464965 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b77ed77-0fc7-4664-bf74-09aeca9922d0 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:11:48.478495 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1280/2592] 10.222.0.46 () {58 vars in 1121 bytes} [Thu Jun 23 16:11:48 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 16:11:48.479655 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75e5a1f-e2d2-4795-9802-6edd092395af admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:11:48.490907 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75e5a1f-e2d2-4795-9802-6edd092395af admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:11:48.495892 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3478bc-76ea-4730-abb2-cd3a1e44f11d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:11:48.496828 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3478bc-76ea-4730-abb2-cd3a1e44f11d admin admin] REQUEST_METHOD: `PUT` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:11:48.497341 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3478bc-76ea-4730-abb2-cd3a1e44f11d admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:11:48.497498 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3478bc-76ea-4730-abb2-cd3a1e44f11d admin admin] PATH_INFO: `/v3/projects/1a2e7d6e4c3d419ca16bfc5cadb48b25/users/343d3862f2444365876beceb62ce74e4/roles/e2a5bbf3200a4d4091d72bc7ff78c34d` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:11:48.516292 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3478bc-76ea-4730-abb2-cd3a1e44f11d admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a2e7d6e4c3d419ca16bfc5cadb48b25, user_id=343d3862f2444365876beceb62ce74e4, role_id=e2a5bbf3200a4d4091d72bc7ff78c34d)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:11:48.519058 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3478bc-76ea-4730-abb2-cd3a1e44f11d admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:11:48.537566 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1313/2593] 10.222.0.46 () {60 vars in 1482 bytes} [Thu Jun 23 16:11:48 2022] PUT /identity/v3/projects/1a2e7d6e4c3d419ca16bfc5cadb48b25/users/343d3862f2444365876beceb62ce74e4/roles/e2a5bbf3200a4d4091d72bc7ff78c34d => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 16:11:48.555359 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff868ae-4810-4f8f-8720-6985baca773a None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:11:48.555874 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff868ae-4810-4f8f-8720-6985baca773a None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:11:48.556461 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff868ae-4810-4f8f-8720-6985baca773a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:11:48.585092 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-4ff868ae-4810-4f8f-8720-6985baca773a None None] MFA Rules not processed for user `343d3862f2444365876beceb62ce74e4`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:11:48.586323 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-4ff868ae-4810-4f8f-8720-6985baca773a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:11:48.601779 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:11:48.656336 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1281/2594] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:11:48 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 16:11:48.661192 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3478bc-76ea-4730-abb2-cd3a1e44f11d admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:11:48.716245 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:11:48.733446 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3478bc-76ea-4730-abb2-cd3a1e44f11d admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:11:48.745352 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d340773-7728-40de-9e5f-9907851259a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:11:48.745352 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d340773-7728-40de-9e5f-9907851259a8 admin admin] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:11:48.745352 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d340773-7728-40de-9e5f-9907851259a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:11:48.745352 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d340773-7728-40de-9e5f-9907851259a8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:11:48.745707 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d340773-7728-40de-9e5f-9907851259a8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:11:48.745707 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d340773-7728-40de-9e5f-9907851259a8 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:11:48.772764 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1314/2595] 10.222.0.46 () {60 vars in 1152 bytes} [Thu Jun 23 16:11:48 2022] POST /identity/v3/projects => generated 383 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 23 16:11:48.777469 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff868ae-4810-4f8f-8720-6985baca773a None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:11:48.789366 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff868ae-4810-4f8f-8720-6985baca773a None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:11:48.848184 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f913a8a-cd62-4c75-855f-f7b141dba898 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:11:48.848869 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f913a8a-cd62-4c75-855f-f7b141dba898 admin admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:11:48.849159 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f913a8a-cd62-4c75-855f-f7b141dba898 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:11:48.849458 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f913a8a-cd62-4c75-855f-f7b141dba898 admin admin] PATH_INFO: `/v3/users` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:11:48.850169 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f913a8a-cd62-4c75-855f-f7b141dba898 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:11:48.852164 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f913a8a-cd62-4c75-855f-f7b141dba898 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:11:48.872690 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1282/2596] 10.222.0.46 () {60 vars in 1143 bytes} [Thu Jun 23 16:11:48 2022] POST /identity/v3/users => generated 298 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 16:11:48.877104 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d340773-7728-40de-9e5f-9907851259a8 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:11:48.887800 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d340773-7728-40de-9e5f-9907851259a8 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:11:48.892490 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79077cdd-6f38-4945-b475-af76fb1d99c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:11:48.893150 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79077cdd-6f38-4945-b475-af76fb1d99c6 admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:11:48.893440 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79077cdd-6f38-4945-b475-af76fb1d99c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:11:48.893722 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79077cdd-6f38-4945-b475-af76fb1d99c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:11:48.894482 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79077cdd-6f38-4945-b475-af76fb1d99c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:11:48.896139 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79077cdd-6f38-4945-b475-af76fb1d99c6 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:11:48.906984 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1315/2597] 10.222.0.46 () {58 vars in 1121 bytes} [Thu Jun 23 16:11:48 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 16:11:48.911629 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f913a8a-cd62-4c75-855f-f7b141dba898 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:11:48.921666 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f913a8a-cd62-4c75-855f-f7b141dba898 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:11:48.927056 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00823ec2-f8a3-489e-867a-1e784b8601c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:11:48.927941 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00823ec2-f8a3-489e-867a-1e784b8601c0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:11:48.928229 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00823ec2-f8a3-489e-867a-1e784b8601c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:11:48.928508 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00823ec2-f8a3-489e-867a-1e784b8601c0 admin admin] PATH_INFO: `/v3/projects/e374711c7edc47dfb75a67af74d449e9/users/c913f5fe84784a7698daa88b5bdf1dd7/roles/812d3c567c52469cb2b35709fbdc857c` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:11:48.943095 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00823ec2-f8a3-489e-867a-1e784b8601c0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e374711c7edc47dfb75a67af74d449e9, user_id=c913f5fe84784a7698daa88b5bdf1dd7, role_id=812d3c567c52469cb2b35709fbdc857c)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:11:48.945769 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00823ec2-f8a3-489e-867a-1e784b8601c0 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:11:48.958606 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1283/2598] 10.222.0.46 () {60 vars in 1482 bytes} [Thu Jun 23 16:11:48 2022] PUT /identity/v3/projects/e374711c7edc47dfb75a67af74d449e9/users/c913f5fe84784a7698daa88b5bdf1dd7/roles/812d3c567c52469cb2b35709fbdc857c => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 16:11:48.962517 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79077cdd-6f38-4945-b475-af76fb1d99c6 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:11:48.973133 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79077cdd-6f38-4945-b475-af76fb1d99c6 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:11:49.042049 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5c747d-ebb7-42e5-ad7b-c76e1b038b10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:11:49.042636 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5c747d-ebb7-42e5-ad7b-c76e1b038b10 admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:11:49.042957 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5c747d-ebb7-42e5-ad7b-c76e1b038b10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:11:49.043235 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5c747d-ebb7-42e5-ad7b-c76e1b038b10 admin admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:11:49.043970 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5c747d-ebb7-42e5-ad7b-c76e1b038b10 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:11:49.045579 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5c747d-ebb7-42e5-ad7b-c76e1b038b10 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:11:49.055546 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1316/2599] 10.222.0.46 () {58 vars in 1121 bytes} [Thu Jun 23 16:11:48 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 16:11:49.059725 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00823ec2-f8a3-489e-867a-1e784b8601c0 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:11:49.070395 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00823ec2-f8a3-489e-867a-1e784b8601c0 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:11:49.075182 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8299f9c-a0b1-46bb-a83f-be977d366392 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:11:49.076072 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8299f9c-a0b1-46bb-a83f-be977d366392 admin admin] REQUEST_METHOD: `PUT` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:11:49.076358 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8299f9c-a0b1-46bb-a83f-be977d366392 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:11:49.076637 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8299f9c-a0b1-46bb-a83f-be977d366392 admin admin] PATH_INFO: `/v3/projects/e374711c7edc47dfb75a67af74d449e9/users/c913f5fe84784a7698daa88b5bdf1dd7/roles/812d3c567c52469cb2b35709fbdc857c` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:11:49.078644 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8299f9c-a0b1-46bb-a83f-be977d366392 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e374711c7edc47dfb75a67af74d449e9, user_id=c913f5fe84784a7698daa88b5bdf1dd7, role_id=812d3c567c52469cb2b35709fbdc857c)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:11:49.081313 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8299f9c-a0b1-46bb-a83f-be977d366392 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:11:49.092208 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1284/2600] 10.222.0.46 () {60 vars in 1482 bytes} [Thu Jun 23 16:11:49 2022] PUT /identity/v3/projects/e374711c7edc47dfb75a67af74d449e9/users/c913f5fe84784a7698daa88b5bdf1dd7/roles/812d3c567c52469cb2b35709fbdc857c => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 16:11:49.103427 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0f23d5-93b6-4a62-ad91-b6e07710b89c None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:11:49.103719 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0f23d5-93b6-4a62-ad91-b6e07710b89c None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:11:49.104005 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0f23d5-93b6-4a62-ad91-b6e07710b89c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:11:49.130491 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-cb0f23d5-93b6-4a62-ad91-b6e07710b89c None None] MFA Rules not processed for user `c913f5fe84784a7698daa88b5bdf1dd7`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:11:49.131739 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-cb0f23d5-93b6-4a62-ad91-b6e07710b89c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:11:49.201464 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1317/2601] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:11:49 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 16:11:49.212541 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8299f9c-a0b1-46bb-a83f-be977d366392 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:11:49.214366 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-d8299f9c-a0b1-46bb-a83f-be977d366392 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:11:49.273434 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8299f9c-a0b1-46bb-a83f-be977d366392 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:11:49.277358 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214c02f3-6e4f-4135-b575-05810e150b17 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce18eaaba3a6477684fff1fd98ec8b51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:11:49.278014 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214c02f3-6e4f-4135-b575-05810e150b17 service cinder] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:11:49.278303 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214c02f3-6e4f-4135-b575-05810e150b17 service cinder] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:11:49.278585 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214c02f3-6e4f-4135-b575-05810e150b17 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:11:49.288120 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214c02f3-6e4f-4135-b575-05810e150b17 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:11:49.289547 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214c02f3-6e4f-4135-b575-05810e150b17 service cinder] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:11:49.299135 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1285/2602] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jun 23 16:11:49 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 16:11:49.395287 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0f23d5-93b6-4a62-ad91-b6e07710b89c None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:11:49.405546 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0f23d5-93b6-4a62-ad91-b6e07710b89c None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:11:49.409370 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9d44ef-a27c-4a88-832a-5686274702b1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce18eaaba3a6477684fff1fd98ec8b51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:11:49.410016 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9d44ef-a27c-4a88-832a-5686274702b1 service cinder] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:11:49.410295 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9d44ef-a27c-4a88-832a-5686274702b1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:11:49.410566 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9d44ef-a27c-4a88-832a-5686274702b1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:11:49.454146 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:11:49.471027 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9d44ef-a27c-4a88-832a-5686274702b1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:11:49.472467 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9d44ef-a27c-4a88-832a-5686274702b1 service cinder] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:11:49.482448 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1318/2603] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jun 23 16:11:49 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 16:11:52.039668 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214c02f3-6e4f-4135-b575-05810e150b17 service cinder] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:11:52.051744 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:11:52.101772 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214c02f3-6e4f-4135-b575-05810e150b17 service cinder] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:11:52.106255 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd24bd3-462f-4e95-bf58-4d4c055413ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:11:52.107146 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd24bd3-462f-4e95-bf58-4d4c055413ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:11:52.107426 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd24bd3-462f-4e95-bf58-4d4c055413ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:11:52.107697 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd24bd3-462f-4e95-bf58-4d4c055413ce admin admin] PATH_INFO: `/v3/users/343d3862f2444365876beceb62ce74e4` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:11:52.108862 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd24bd3-462f-4e95-bf58-4d4c055413ce admin admin] RBAC: Authorizing `identity:delete_user(user_id=343d3862f2444365876beceb62ce74e4)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:11:52.110918 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd24bd3-462f-4e95-bf58-4d4c055413ce admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:11:52.157667 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-cfd24bd3-462f-4e95-bf58-4d4c055413ce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '343d3862f2444365876beceb62ce74e4'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:11:52.158318 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-cfd24bd3-462f-4e95-bf58-4d4c055413ce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '343d3862f2444365876beceb62ce74e4'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:11:52.165100 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-cfd24bd3-462f-4e95-bf58-4d4c055413ce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '343d3862f2444365876beceb62ce74e4'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:11:52.172301 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-cfd24bd3-462f-4e95-bf58-4d4c055413ce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '343d3862f2444365876beceb62ce74e4'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:11:52.172933 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-cfd24bd3-462f-4e95-bf58-4d4c055413ce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '343d3862f2444365876beceb62ce74e4'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:11:52.193160 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1286/2604] 10.222.0.46 () {58 vars in 1223 bytes} [Thu Jun 23 16:11:52 2022] DELETE /identity/v3/users/343d3862f2444365876beceb62ce74e4 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 16:11:52.197761 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7458736f-2048-4d40-9929-ba46fa4c8396 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:11:52.198259 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7458736f-2048-4d40-9929-ba46fa4c8396 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:11:52.198358 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7458736f-2048-4d40-9929-ba46fa4c8396 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:11:52.215132 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-7458736f-2048-4d40-9929-ba46fa4c8396 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:11:52.216339 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-7458736f-2048-4d40-9929-ba46fa4c8396 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:11:52.278638 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1319/2605] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:11:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 16:11:52.289160 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd24bd3-462f-4e95-bf58-4d4c055413ce admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:11:52.290580 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-cfd24bd3-462f-4e95-bf58-4d4c055413ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:11:52.347527 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd24bd3-462f-4e95-bf58-4d4c055413ce admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:11:52.351839 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf0e3a4-3c19-46fb-8ed9-044a43f892c3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51784064951a4619ae09b9167fb0fd0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:11:52.352492 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf0e3a4-3c19-46fb-8ed9-044a43f892c3 service neutron] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:11:52.352778 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf0e3a4-3c19-46fb-8ed9-044a43f892c3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:11:52.353056 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf0e3a4-3c19-46fb-8ed9-044a43f892c3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:11:52.362067 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf0e3a4-3c19-46fb-8ed9-044a43f892c3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:11:52.363560 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf0e3a4-3c19-46fb-8ed9-044a43f892c3 service neutron] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:11:52.372986 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1287/2606] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jun 23 16:11:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 16:11:52.641665 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7458736f-2048-4d40-9929-ba46fa4c8396 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:11:52.643173 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-7458736f-2048-4d40-9929-ba46fa4c8396 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:11:52.657671 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7458736f-2048-4d40-9929-ba46fa4c8396 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:11:52.662991 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65795d42-4113-44e6-aba4-741e0030501b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:11:52.664167 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65795d42-4113-44e6-aba4-741e0030501b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:11:52.664346 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65795d42-4113-44e6-aba4-741e0030501b admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:11:52.664679 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65795d42-4113-44e6-aba4-741e0030501b admin admin] PATH_INFO: `/v3/projects/1a2e7d6e4c3d419ca16bfc5cadb48b25` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:11:52.665908 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65795d42-4113-44e6-aba4-741e0030501b admin admin] RBAC: Authorizing `identity:delete_project(project_id=1a2e7d6e4c3d419ca16bfc5cadb48b25)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:11:52.668208 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65795d42-4113-44e6-aba4-741e0030501b admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:11:52.716773 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-65795d42-4113-44e6-aba4-741e0030501b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1a2e7d6e4c3d419ca16bfc5cadb48b25', 'request_id': 'req-65795d42-4113-44e6-aba4-741e0030501b'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:11:52.722359 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-65795d42-4113-44e6-aba4-741e0030501b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1a2e7d6e4c3d419ca16bfc5cadb48b25', 'request_id': 'req-65795d42-4113-44e6-aba4-741e0030501b'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:11:52.728588 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-65795d42-4113-44e6-aba4-741e0030501b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1a2e7d6e4c3d419ca16bfc5cadb48b25', 'request_id': 'req-65795d42-4113-44e6-aba4-741e0030501b'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:11:52.741080 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-65795d42-4113-44e6-aba4-741e0030501b admin admin] The token cache is being invalidate because project 1a2e7d6e4c3d419ca16bfc5cadb48b25 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80153) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 16:11:52.741376 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-65795d42-4113-44e6-aba4-741e0030501b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:11:52.742071 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-65795d42-4113-44e6-aba4-741e0030501b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:11:52.743700 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1320/2607] 10.222.0.46 () {58 vars in 1232 bytes} [Thu Jun 23 16:11:52 2022] DELETE /identity/v3/projects/1a2e7d6e4c3d419ca16bfc5cadb48b25 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 16:11:52.747722 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf0e3a4-3c19-46fb-8ed9-044a43f892c3 service neutron] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:11:52.757824 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf0e3a4-3c19-46fb-8ed9-044a43f892c3 service neutron] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:11:52.788351 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:11:52.816624 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38df1526-3ba5-403f-b4c2-ef23362d5bd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:11:52.818033 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38df1526-3ba5-403f-b4c2-ef23362d5bd9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:11:52.818319 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38df1526-3ba5-403f-b4c2-ef23362d5bd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:11:52.818598 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38df1526-3ba5-403f-b4c2-ef23362d5bd9 admin admin] PATH_INFO: `/v3/users/c913f5fe84784a7698daa88b5bdf1dd7` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:11:52.819872 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38df1526-3ba5-403f-b4c2-ef23362d5bd9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c913f5fe84784a7698daa88b5bdf1dd7)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:11:52.821810 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38df1526-3ba5-403f-b4c2-ef23362d5bd9 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:11:52.870052 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-38df1526-3ba5-403f-b4c2-ef23362d5bd9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c913f5fe84784a7698daa88b5bdf1dd7'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:11:52.870708 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-38df1526-3ba5-403f-b4c2-ef23362d5bd9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c913f5fe84784a7698daa88b5bdf1dd7'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:11:52.877320 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-38df1526-3ba5-403f-b4c2-ef23362d5bd9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c913f5fe84784a7698daa88b5bdf1dd7'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:11:52.887507 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-38df1526-3ba5-403f-b4c2-ef23362d5bd9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c913f5fe84784a7698daa88b5bdf1dd7'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:11:52.888304 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-38df1526-3ba5-403f-b4c2-ef23362d5bd9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c913f5fe84784a7698daa88b5bdf1dd7'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:11:52.906645 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1288/2608] 10.222.0.46 () {58 vars in 1223 bytes} [Thu Jun 23 16:11:52 2022] DELETE /identity/v3/users/c913f5fe84784a7698daa88b5bdf1dd7 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 16:11:53.187879 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65795d42-4113-44e6-aba4-741e0030501b admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:11:53.189534 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-65795d42-4113-44e6-aba4-741e0030501b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:11:53.204995 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65795d42-4113-44e6-aba4-741e0030501b admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:11:53.263666 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81727d46-df36-41f7-a979-74a58f8a6a3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:11:53.264568 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81727d46-df36-41f7-a979-74a58f8a6a3c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:11:53.264844 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81727d46-df36-41f7-a979-74a58f8a6a3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:11:53.265115 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81727d46-df36-41f7-a979-74a58f8a6a3c admin admin] PATH_INFO: `/v3/projects/e374711c7edc47dfb75a67af74d449e9` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:11:53.266312 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81727d46-df36-41f7-a979-74a58f8a6a3c admin admin] RBAC: Authorizing `identity:delete_project(project_id=e374711c7edc47dfb75a67af74d449e9)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:11:53.268331 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81727d46-df36-41f7-a979-74a58f8a6a3c admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:11:53.276925 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:11:53.316989 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-81727d46-df36-41f7-a979-74a58f8a6a3c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e374711c7edc47dfb75a67af74d449e9', 'request_id': 'req-81727d46-df36-41f7-a979-74a58f8a6a3c'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:11:53.322632 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-81727d46-df36-41f7-a979-74a58f8a6a3c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e374711c7edc47dfb75a67af74d449e9', 'request_id': 'req-81727d46-df36-41f7-a979-74a58f8a6a3c'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:11:53.328889 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-81727d46-df36-41f7-a979-74a58f8a6a3c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e374711c7edc47dfb75a67af74d449e9', 'request_id': 'req-81727d46-df36-41f7-a979-74a58f8a6a3c'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:11:53.341931 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-81727d46-df36-41f7-a979-74a58f8a6a3c admin admin] The token cache is being invalidate because project e374711c7edc47dfb75a67af74d449e9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80153) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 16:11:53.342211 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-81727d46-df36-41f7-a979-74a58f8a6a3c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:11:53.342882 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-81727d46-df36-41f7-a979-74a58f8a6a3c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:11:53.344469 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1321/2609] 10.222.0.46 () {58 vars in 1232 bytes} [Thu Jun 23 16:11:53 2022] DELETE /identity/v3/projects/e374711c7edc47dfb75a67af74d449e9 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 16:11:53.351195 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e560272-88b8-493d-8ae8-e70249d92a98 None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:11:53.351678 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e560272-88b8-493d-8ae8-e70249d92a98 None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:11:53.351813 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e560272-88b8-493d-8ae8-e70249d92a98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:11:53.369700 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-4e560272-88b8-493d-8ae8-e70249d92a98 None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:11:53.370860 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-4e560272-88b8-493d-8ae8-e70249d92a98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:11:53.431039 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1289/2610] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:11:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 16:11:53.438832 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79845ed3-8706-4fc8-8c46-6e7fde8f2c9a None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:11:53.439124 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79845ed3-8706-4fc8-8c46-6e7fde8f2c9a None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:11:53.439406 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79845ed3-8706-4fc8-8c46-6e7fde8f2c9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:11:53.457771 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-79845ed3-8706-4fc8-8c46-6e7fde8f2c9a None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:11:53.459025 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-79845ed3-8706-4fc8-8c46-6e7fde8f2c9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:11:53.466987 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1322/2611] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:11:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 16:11:53.471133 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e560272-88b8-493d-8ae8-e70249d92a98 None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:11:53.481385 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e560272-88b8-493d-8ae8-e70249d92a98 None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:11:53.485944 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe11911-f6b4-4fd0-b3c7-8b1c85d8ce15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:11:53.486602 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe11911-f6b4-4fd0-b3c7-8b1c85d8ce15 admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:11:53.486922 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe11911-f6b4-4fd0-b3c7-8b1c85d8ce15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:11:53.487199 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe11911-f6b4-4fd0-b3c7-8b1c85d8ce15 admin admin] PATH_INFO: `/v3/domains` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:11:53.487928 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe11911-f6b4-4fd0-b3c7-8b1c85d8ce15 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80154) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 16:11:53.488260 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe11911-f6b4-4fd0-b3c7-8b1c85d8ce15 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:11:53.490412 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe11911-f6b4-4fd0-b3c7-8b1c85d8ce15 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:11:53.502051 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1290/2612] 10.222.0.46 () {58 vars in 1152 bytes} [Thu Jun 23 16:11:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 16:11:53.506319 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79845ed3-8706-4fc8-8c46-6e7fde8f2c9a None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:11:53.515751 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79845ed3-8706-4fc8-8c46-6e7fde8f2c9a None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:11:53.520493 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a6d0f8-806f-40a3-bf75-8fb7744dfc17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:11:53.521150 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a6d0f8-806f-40a3-bf75-8fb7744dfc17 admin admin] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:11:53.521468 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a6d0f8-806f-40a3-bf75-8fb7744dfc17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:11:53.521750 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a6d0f8-806f-40a3-bf75-8fb7744dfc17 admin admin] PATH_INFO: `/v3/projects` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:11:53.522458 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a6d0f8-806f-40a3-bf75-8fb7744dfc17 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:11:53.524671 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a6d0f8-806f-40a3-bf75-8fb7744dfc17 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:11:53.550469 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1323/2613] 10.222.0.46 () {60 vars in 1152 bytes} [Thu Jun 23 16:11:53 2022] POST /identity/v3/projects => generated 369 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 16:11:53.557618 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe11911-f6b4-4fd0-b3c7-8b1c85d8ce15 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:11:53.567872 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe11911-f6b4-4fd0-b3c7-8b1c85d8ce15 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:11:53.605612 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:11:53.626178 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21764ef3-1011-4810-9b9f-fcff8587f997 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:11:53.626843 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21764ef3-1011-4810-9b9f-fcff8587f997 admin admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:11:53.627136 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21764ef3-1011-4810-9b9f-fcff8587f997 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:11:53.627418 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21764ef3-1011-4810-9b9f-fcff8587f997 admin admin] PATH_INFO: `/v3/users` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:11:53.628108 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21764ef3-1011-4810-9b9f-fcff8587f997 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:11:53.630022 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21764ef3-1011-4810-9b9f-fcff8587f997 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:11:53.649979 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1291/2614] 10.222.0.46 () {60 vars in 1143 bytes} [Thu Jun 23 16:11:53 2022] POST /identity/v3/users => generated 285 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 16:11:53.654113 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a6d0f8-806f-40a3-bf75-8fb7744dfc17 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:11:53.665225 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a6d0f8-806f-40a3-bf75-8fb7744dfc17 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:11:53.670274 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771b830f-3d69-4b70-b107-8f405b26f774 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:11:53.671092 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771b830f-3d69-4b70-b107-8f405b26f774 admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:11:53.671379 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771b830f-3d69-4b70-b107-8f405b26f774 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:11:53.671659 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771b830f-3d69-4b70-b107-8f405b26f774 admin admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:11:53.672392 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771b830f-3d69-4b70-b107-8f405b26f774 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:11:53.673992 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771b830f-3d69-4b70-b107-8f405b26f774 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:11:53.684959 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1324/2615] 10.222.0.46 () {58 vars in 1121 bytes} [Thu Jun 23 16:11:53 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 16:11:53.688952 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21764ef3-1011-4810-9b9f-fcff8587f997 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:11:53.698812 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21764ef3-1011-4810-9b9f-fcff8587f997 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:11:53.703409 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6c1010-73ff-43b0-8ed6-44b7346ad013 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:11:53.704062 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6c1010-73ff-43b0-8ed6-44b7346ad013 admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:11:53.704349 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6c1010-73ff-43b0-8ed6-44b7346ad013 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:11:53.704627 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6c1010-73ff-43b0-8ed6-44b7346ad013 admin admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:11:53.705352 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6c1010-73ff-43b0-8ed6-44b7346ad013 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:11:53.706988 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6c1010-73ff-43b0-8ed6-44b7346ad013 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:11:53.717301 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1292/2616] 10.222.0.46 () {58 vars in 1121 bytes} [Thu Jun 23 16:11:53 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 16:11:53.721289 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771b830f-3d69-4b70-b107-8f405b26f774 admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:11:53.731982 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771b830f-3d69-4b70-b107-8f405b26f774 admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:11:53.736659 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86848a3-dae3-4af2-be9b-657c11ffc81d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:11:53.737546 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86848a3-dae3-4af2-be9b-657c11ffc81d admin admin] REQUEST_METHOD: `PUT` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:11:53.737834 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86848a3-dae3-4af2-be9b-657c11ffc81d admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:11:53.738178 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86848a3-dae3-4af2-be9b-657c11ffc81d admin admin] PATH_INFO: `/v3/projects/0554d48d2d6d47678942b8b4f8277f49/users/706077483df14bccb4a527dd6dbe6d34/roles/e2a5bbf3200a4d4091d72bc7ff78c34d` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:11:53.752597 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86848a3-dae3-4af2-be9b-657c11ffc81d admin admin] RBAC: Authorizing `identity:create_grant(project_id=0554d48d2d6d47678942b8b4f8277f49, user_id=706077483df14bccb4a527dd6dbe6d34, role_id=e2a5bbf3200a4d4091d72bc7ff78c34d)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:11:53.755331 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86848a3-dae3-4af2-be9b-657c11ffc81d admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:11:53.768127 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1325/2617] 10.222.0.46 () {60 vars in 1482 bytes} [Thu Jun 23 16:11:53 2022] PUT /identity/v3/projects/0554d48d2d6d47678942b8b4f8277f49/users/706077483df14bccb4a527dd6dbe6d34/roles/e2a5bbf3200a4d4091d72bc7ff78c34d => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 16:11:53.779305 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d2ccc6-7a40-465a-99ba-dde00a6da48a None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:11:53.779599 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d2ccc6-7a40-465a-99ba-dde00a6da48a None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:11:53.779879 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d2ccc6-7a40-465a-99ba-dde00a6da48a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:11:53.807155 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-06d2ccc6-7a40-465a-99ba-dde00a6da48a None None] MFA Rules not processed for user `706077483df14bccb4a527dd6dbe6d34`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:11:53.808405 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-06d2ccc6-7a40-465a-99ba-dde00a6da48a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:11:53.873558 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1293/2618] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:11:53 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 16:11:53.885379 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86848a3-dae3-4af2-be9b-657c11ffc81d admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:11:53.887100 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-a86848a3-dae3-4af2-be9b-657c11ffc81d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:11:53.945144 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86848a3-dae3-4af2-be9b-657c11ffc81d admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:11:53.948832 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22690918-18eb-4c0c-8083-4df3f49687ef service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83970271286e451e9008bbf07cd3fa86', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:11:53.949571 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22690918-18eb-4c0c-8083-4df3f49687ef service glance] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:11:53.949860 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22690918-18eb-4c0c-8083-4df3f49687ef service glance] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:11:53.950144 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22690918-18eb-4c0c-8083-4df3f49687ef service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:11:53.959230 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22690918-18eb-4c0c-8083-4df3f49687ef service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:11:53.960736 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22690918-18eb-4c0c-8083-4df3f49687ef service glance] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:11:53.969712 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1326/2619] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jun 23 16:11:53 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 16:11:55.263570 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d2ccc6-7a40-465a-99ba-dde00a6da48a None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:11:55.265134 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-06d2ccc6-7a40-465a-99ba-dde00a6da48a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:11:55.323635 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d2ccc6-7a40-465a-99ba-dde00a6da48a None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:11:55.327210 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbae0a6-1d6e-4809-86f0-fd4a0007009e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce18eaaba3a6477684fff1fd98ec8b51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:11:55.327973 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbae0a6-1d6e-4809-86f0-fd4a0007009e service cinder] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:11:55.328336 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbae0a6-1d6e-4809-86f0-fd4a0007009e service cinder] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:11:55.328688 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbae0a6-1d6e-4809-86f0-fd4a0007009e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:11:55.330893 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-0cbae0a6-1d6e-4809-86f0-fd4a0007009e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:11:55.402796 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:11:55.421774 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbae0a6-1d6e-4809-86f0-fd4a0007009e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:11:55.423943 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbae0a6-1d6e-4809-86f0-fd4a0007009e service cinder] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:11:55.443371 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1294/2620] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jun 23 16:11:55 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 16:11:59.873338 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22690918-18eb-4c0c-8083-4df3f49687ef service glance] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:11:59.876910 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-22690918-18eb-4c0c-8083-4df3f49687ef service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:11:59.985441 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:12:00.044451 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22690918-18eb-4c0c-8083-4df3f49687ef service glance] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:12:00.052990 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9dc692-0625-4fbe-bb54-a50ae5fe6e95 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '705115d19bab424d894bea8b1b48a7f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:12:00.053589 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9dc692-0625-4fbe-bb54-a50ae5fe6e95 service nova] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:12:00.053981 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9dc692-0625-4fbe-bb54-a50ae5fe6e95 service nova] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:12:00.054259 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9dc692-0625-4fbe-bb54-a50ae5fe6e95 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:12:00.057197 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-ae9dc692-0625-4fbe-bb54-a50ae5fe6e95 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:12:00.074466 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9dc692-0625-4fbe-bb54-a50ae5fe6e95 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:12:00.076124 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9dc692-0625-4fbe-bb54-a50ae5fe6e95 service nova] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:12:00.093060 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1327/2621] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jun 23 16:11:59 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 16:12:03.728575 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbae0a6-1d6e-4809-86f0-fd4a0007009e service cinder] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:12:03.730599 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-0cbae0a6-1d6e-4809-86f0-fd4a0007009e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:12:03.789253 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbae0a6-1d6e-4809-86f0-fd4a0007009e service cinder] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:12:03.792762 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0917e1ee-294d-4e45-9d8d-f1f2d66aa1da service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51784064951a4619ae09b9167fb0fd0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:12:03.793484 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0917e1ee-294d-4e45-9d8d-f1f2d66aa1da service neutron] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:12:03.793893 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0917e1ee-294d-4e45-9d8d-f1f2d66aa1da service neutron] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:12:03.794181 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0917e1ee-294d-4e45-9d8d-f1f2d66aa1da service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:12:03.796148 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-0917e1ee-294d-4e45-9d8d-f1f2d66aa1da service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:12:03.882027 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0917e1ee-294d-4e45-9d8d-f1f2d66aa1da service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:12:03.883653 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0917e1ee-294d-4e45-9d8d-f1f2d66aa1da service neutron] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:12:03.894739 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1295/2622] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jun 23 16:12:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 16:12:07.308360 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9dc692-0625-4fbe-bb54-a50ae5fe6e95 service nova] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:12:07.310419 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-ae9dc692-0625-4fbe-bb54-a50ae5fe6e95 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:12:07.336697 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9dc692-0625-4fbe-bb54-a50ae5fe6e95 service nova] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:12:07.342104 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7007e55e-5113-44ad-ac93-ea70653b5002 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:12:07.343185 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7007e55e-5113-44ad-ac93-ea70653b5002 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:12:07.343579 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7007e55e-5113-44ad-ac93-ea70653b5002 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:12:07.343945 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7007e55e-5113-44ad-ac93-ea70653b5002 admin admin] PATH_INFO: `/v3/users/5dd4357556944c7089efb34ee58a8e04` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:12:07.345265 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7007e55e-5113-44ad-ac93-ea70653b5002 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5dd4357556944c7089efb34ee58a8e04)` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:12:07.347577 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7007e55e-5113-44ad-ac93-ea70653b5002 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:12:07.420189 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-7007e55e-5113-44ad-ac93-ea70653b5002 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5dd4357556944c7089efb34ee58a8e04'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:12:07.429338 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-7007e55e-5113-44ad-ac93-ea70653b5002 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5dd4357556944c7089efb34ee58a8e04'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:12:07.430024 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-7007e55e-5113-44ad-ac93-ea70653b5002 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5dd4357556944c7089efb34ee58a8e04'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:12:07.430804 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-7007e55e-5113-44ad-ac93-ea70653b5002 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5dd4357556944c7089efb34ee58a8e04'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:12:07.446381 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.notifications [None req-7007e55e-5113-44ad-ac93-ea70653b5002 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5dd4357556944c7089efb34ee58a8e04'} {{(pid=80153) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:12:07.468744 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1328/2623] 10.222.0.46 () {58 vars in 1223 bytes} [Thu Jun 23 16:12:07 2022] DELETE /identity/v3/users/5dd4357556944c7089efb34ee58a8e04 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 16:12:07.647843 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0917e1ee-294d-4e45-9d8d-f1f2d66aa1da service neutron] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:12:07.668520 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0917e1ee-294d-4e45-9d8d-f1f2d66aa1da service neutron] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:12:07.763388 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8d1e5c-5361-479e-b786-49008aa376c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:12:07.765122 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8d1e5c-5361-479e-b786-49008aa376c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:12:07.765122 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8d1e5c-5361-479e-b786-49008aa376c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:12:07.765463 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8d1e5c-5361-479e-b786-49008aa376c3 admin admin] PATH_INFO: `/v3/projects/7412ae4990fa4fe782cc4c5e3db00d75` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:12:07.768205 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8d1e5c-5361-479e-b786-49008aa376c3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7412ae4990fa4fe782cc4c5e3db00d75)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:12:07.770590 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8d1e5c-5361-479e-b786-49008aa376c3 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:12:07.820968 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:12:07.847628 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-6a8d1e5c-5361-479e-b786-49008aa376c3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7412ae4990fa4fe782cc4c5e3db00d75', 'request_id': 'req-6a8d1e5c-5361-479e-b786-49008aa376c3'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:12:07.854279 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-6a8d1e5c-5361-479e-b786-49008aa376c3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7412ae4990fa4fe782cc4c5e3db00d75', 'request_id': 'req-6a8d1e5c-5361-479e-b786-49008aa376c3'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:12:07.865033 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-6a8d1e5c-5361-479e-b786-49008aa376c3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7412ae4990fa4fe782cc4c5e3db00d75', 'request_id': 'req-6a8d1e5c-5361-479e-b786-49008aa376c3'} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:12:07.880961 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-6a8d1e5c-5361-479e-b786-49008aa376c3 admin admin] The token cache is being invalidate because project 7412ae4990fa4fe782cc4c5e3db00d75 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80154) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 23 16:12:07.882058 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-6a8d1e5c-5361-479e-b786-49008aa376c3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:12:07.882892 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.notifications [None req-6a8d1e5c-5361-479e-b786-49008aa376c3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80154) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 23 16:12:07.884946 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1296/2624] 10.222.0.46 () {58 vars in 1232 bytes} [Thu Jun 23 16:12:07 2022] DELETE /identity/v3/projects/7412ae4990fa4fe782cc4c5e3db00d75 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 16:12:07.889782 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ad3aab-ce6f-4a16-98b5-bf5cabf8106a None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:12:07.890081 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ad3aab-ce6f-4a16-98b5-bf5cabf8106a None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:12:07.890342 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ad3aab-ce6f-4a16-98b5-bf5cabf8106a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:12:07.910253 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-12ad3aab-ce6f-4a16-98b5-bf5cabf8106a None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:12:07.911425 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-12ad3aab-ce6f-4a16-98b5-bf5cabf8106a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:12:07.978878 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1329/2625] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:12:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 16:12:07.987247 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f571724-777b-4105-9946-f8d7ed7808ff None None] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:12:07.987537 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f571724-777b-4105-9946-f8d7ed7808ff None None] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:12:07.988245 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f571724-777b-4105-9946-f8d7ed7808ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:12:08.008803 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.auth.core [None req-0f571724-777b-4105-9946-f8d7ed7808ff None None] MFA Rules not processed for user `e9c7fd1e1ef54284a2b594ce3d58fc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80154) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:12:08.009941 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-0f571724-777b-4105-9946-f8d7ed7808ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:12:08.018578 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1297/2626] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:12:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 23 16:12:08.023007 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ad3aab-ce6f-4a16-98b5-bf5cabf8106a None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:12:08.033058 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ad3aab-ce6f-4a16-98b5-bf5cabf8106a None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:12:08.037637 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4813946d-ea20-46a7-a003-d58cb712cace admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:12:08.038332 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4813946d-ea20-46a7-a003-d58cb712cace admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:12:08.038610 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4813946d-ea20-46a7-a003-d58cb712cace admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:12:08.038922 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4813946d-ea20-46a7-a003-d58cb712cace admin admin] PATH_INFO: `/v3/domains` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:12:08.039648 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4813946d-ea20-46a7-a003-d58cb712cace admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80153) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 23 16:12:08.039952 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4813946d-ea20-46a7-a003-d58cb712cace admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:12:08.041544 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4813946d-ea20-46a7-a003-d58cb712cace admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:12:08.054359 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1330/2627] 10.222.0.46 () {58 vars in 1152 bytes} [Thu Jun 23 16:12:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 23 16:12:08.058785 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f571724-777b-4105-9946-f8d7ed7808ff None None] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:12:08.073831 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f571724-777b-4105-9946-f8d7ed7808ff None None] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:12:08.079610 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b729e6-f547-4923-9049-043ece3d6511 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:12:08.080384 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b729e6-f547-4923-9049-043ece3d6511 admin admin] REQUEST_METHOD: `POST` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:12:08.081159 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b729e6-f547-4923-9049-043ece3d6511 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:12:08.081159 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b729e6-f547-4923-9049-043ece3d6511 admin admin] PATH_INFO: `/v3/projects` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:12:08.081802 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b729e6-f547-4923-9049-043ece3d6511 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:12:08.086045 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b729e6-f547-4923-9049-043ece3d6511 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:12:08.138395 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1298/2628] 10.222.0.46 () {60 vars in 1152 bytes} [Thu Jun 23 16:12:08 2022] POST /identity/v3/projects => generated 371 bytes in 80 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 16:12:08.142569 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4813946d-ea20-46a7-a003-d58cb712cace admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:12:08.154145 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4813946d-ea20-46a7-a003-d58cb712cace admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:12:08.175665 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:12:08.223898 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cdc07c-7a48-4b3f-b13b-fbbeb5c51f9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:12:08.224639 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cdc07c-7a48-4b3f-b13b-fbbeb5c51f9e admin admin] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:12:08.224927 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cdc07c-7a48-4b3f-b13b-fbbeb5c51f9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:12:08.225752 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cdc07c-7a48-4b3f-b13b-fbbeb5c51f9e admin admin] PATH_INFO: `/v3/users` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:12:08.226454 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7cdc07c-7a48-4b3f-b13b-fbbeb5c51f9e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:12:08.228445 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7cdc07c-7a48-4b3f-b13b-fbbeb5c51f9e admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:12:08.254233 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1331/2629] 10.222.0.46 () {60 vars in 1143 bytes} [Thu Jun 23 16:12:08 2022] POST /identity/v3/users => generated 286 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 23 16:12:08.255445 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b729e6-f547-4923-9049-043ece3d6511 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:12:08.266409 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b729e6-f547-4923-9049-043ece3d6511 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:12:08.271265 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea3d9f7-dedd-4c00-a9e1-0e21fff599e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:12:08.272635 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea3d9f7-dedd-4c00-a9e1-0e21fff599e6 admin admin] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:12:08.272950 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea3d9f7-dedd-4c00-a9e1-0e21fff599e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:12:08.273223 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea3d9f7-dedd-4c00-a9e1-0e21fff599e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:12:08.273950 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea3d9f7-dedd-4c00-a9e1-0e21fff599e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:12:08.275934 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea3d9f7-dedd-4c00-a9e1-0e21fff599e6 admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:12:08.286727 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1299/2630] 10.222.0.46 () {58 vars in 1121 bytes} [Thu Jun 23 16:12:08 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 23 16:12:08.290459 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cdc07c-7a48-4b3f-b13b-fbbeb5c51f9e admin admin] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:12:08.300448 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cdc07c-7a48-4b3f-b13b-fbbeb5c51f9e admin admin] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:12:08.306434 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d13d94a-70fc-458c-bb18-5f9001eb88b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:12:08.307142 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d13d94a-70fc-458c-bb18-5f9001eb88b0 admin admin] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:12:08.307422 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d13d94a-70fc-458c-bb18-5f9001eb88b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:12:08.308030 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d13d94a-70fc-458c-bb18-5f9001eb88b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:12:08.308462 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d13d94a-70fc-458c-bb18-5f9001eb88b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:12:08.310320 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d13d94a-70fc-458c-bb18-5f9001eb88b0 admin admin] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:12:08.321379 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1332/2631] 10.222.0.46 () {58 vars in 1121 bytes} [Thu Jun 23 16:12:08 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 23 16:12:08.325616 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea3d9f7-dedd-4c00-a9e1-0e21fff599e6 admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:12:08.336198 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea3d9f7-dedd-4c00-a9e1-0e21fff599e6 admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:12:08.340854 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7eeaef-2ce8-4193-9c92-22a6c09b910e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9c7fd1e1ef54284a2b594ce3d58fc5a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '110dee34db4740f989f715e757ee809c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:12:08.341798 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7eeaef-2ce8-4193-9c92-22a6c09b910e admin admin] REQUEST_METHOD: `PUT` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:12:08.342083 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7eeaef-2ce8-4193-9c92-22a6c09b910e admin admin] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:12:08.342359 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7eeaef-2ce8-4193-9c92-22a6c09b910e admin admin] PATH_INFO: `/v3/projects/6e4b35ae42c143f5a2a5469be0d9201f/users/749a44eba3164fe4b3603d5e202c55a7/roles/e2a5bbf3200a4d4091d72bc7ff78c34d` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:12:08.357592 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7eeaef-2ce8-4193-9c92-22a6c09b910e admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e4b35ae42c143f5a2a5469be0d9201f, user_id=749a44eba3164fe4b3603d5e202c55a7, role_id=e2a5bbf3200a4d4091d72bc7ff78c34d)` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:12:08.360318 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7eeaef-2ce8-4193-9c92-22a6c09b910e admin admin] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:12:08.374781 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1300/2632] 10.222.0.46 () {60 vars in 1482 bytes} [Thu Jun 23 16:12:08 2022] PUT /identity/v3/projects/6e4b35ae42c143f5a2a5469be0d9201f/users/749a44eba3164fe4b3603d5e202c55a7/roles/e2a5bbf3200a4d4091d72bc7ff78c34d => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 23 16:12:08.385795 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ad0abc-c61d-435b-88cc-6361199dbc51 None None] REQUEST_METHOD: `POST` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:12:08.386240 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ad0abc-c61d-435b-88cc-6361199dbc51 None None] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:12:08.386619 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ad0abc-c61d-435b-88cc-6361199dbc51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:12:08.416570 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.auth.core [None req-43ad0abc-c61d-435b-88cc-6361199dbc51 None None] MFA Rules not processed for user `749a44eba3164fe4b3603d5e202c55a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80153) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 23 16:12:08.417890 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-43ad0abc-c61d-435b-88cc-6361199dbc51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:12:08.495889 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1333/2633] 10.222.0.46 () {58 vars in 957 bytes} [Thu Jun 23 16:12:08 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 23 16:12:08.507142 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7eeaef-2ce8-4193-9c92-22a6c09b910e admin admin] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:12:08.509894 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-4e7eeaef-2ce8-4193-9c92-22a6c09b910e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:12:08.596915 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7eeaef-2ce8-4193-9c92-22a6c09b910e admin admin] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:12:08.602992 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd827fb-51a3-4d7d-b1c1-dedb8b617444 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '705115d19bab424d894bea8b1b48a7f0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:12:08.603771 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd827fb-51a3-4d7d-b1c1-dedb8b617444 service nova] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:12:08.604136 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd827fb-51a3-4d7d-b1c1-dedb8b617444 service nova] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:12:08.604497 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd827fb-51a3-4d7d-b1c1-dedb8b617444 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:12:08.614723 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd827fb-51a3-4d7d-b1c1-dedb8b617444 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:12:08.617671 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd827fb-51a3-4d7d-b1c1-dedb8b617444 service nova] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:12:08.632707 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1301/2634] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jun 23 16:12:08 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 16:12:18.492859 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=57,keystone:INSERT=3 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:12:18.624917 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:DELETE=6,keystone:SELECT=75,keystone:INSERT=3 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:14:58.849854 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ad0abc-c61d-435b-88cc-6361199dbc51 None None] Authenticating user token {{(pid=80153) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:14:58.851336 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-43ad0abc-c61d-435b-88cc-6361199dbc51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:14:58.988495 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ad0abc-c61d-435b-88cc-6361199dbc51 None None] Validating token access rules against request {{(pid=80153) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:14:58.993520 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c27aa89-45be-47cc-96f2-5122332f7934 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3f66988b73644fe38b9849663ee70f25', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80153) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:14:58.994200 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c27aa89-45be-47cc-96f2-5122332f7934 service placement] REQUEST_METHOD: `GET` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:14:58.994484 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c27aa89-45be-47cc-96f2-5122332f7934 service placement] SCRIPT_NAME: `/identity` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:14:58.994802 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c27aa89-45be-47cc-96f2-5122332f7934 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80153) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:14:58.996593 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.fernet_utils [None req-5c27aa89-45be-47cc-96f2-5122332f7934 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80153) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:14:59.007898 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c27aa89-45be-47cc-96f2-5122332f7934 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:14:59.009482 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c27aa89-45be-47cc-96f2-5122332f7934 service placement] RBAC: Authorization granted {{(pid=80153) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:14:59.019912 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: [pid: 80153|app: 0|req: 1334/2635] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jun 23 16:14:58 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 23 16:15:09.015691 dv-cinder-835384-9-iscsi devstack@keystone.service[80153]: DEBUG dbcounter [-] [80153] Writing DB stats keystone:SELECT=63 {{(pid=80153) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 23 16:17:04.595815 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd827fb-51a3-4d7d-b1c1-dedb8b617444 service nova] Authenticating user token {{(pid=80154) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 23 16:17:04.597253 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-dbd827fb-51a3-4d7d-b1c1-dedb8b617444 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:17:04.686183 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd827fb-51a3-4d7d-b1c1-dedb8b617444 service nova] Validating token access rules against request {{(pid=80154) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 23 16:17:04.692087 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230aae66-ae81-4e4f-9de6-ca5f906ee29d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51784064951a4619ae09b9167fb0fd0d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '28ecdf5b474c44098c722f677d2a0b77', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80154) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 23 16:17:04.692744 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230aae66-ae81-4e4f-9de6-ca5f906ee29d service neutron] REQUEST_METHOD: `GET` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 23 16:17:04.693031 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230aae66-ae81-4e4f-9de6-ca5f906ee29d service neutron] SCRIPT_NAME: `/identity` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 23 16:17:04.693312 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230aae66-ae81-4e4f-9de6-ca5f906ee29d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80154) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 23 16:17:04.695174 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.fernet_utils [None req-230aae66-ae81-4e4f-9de6-ca5f906ee29d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80154) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 23 16:17:04.704696 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230aae66-ae81-4e4f-9de6-ca5f906ee29d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 23 16:17:04.706109 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230aae66-ae81-4e4f-9de6-ca5f906ee29d service neutron] RBAC: Authorization granted {{(pid=80154) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 23 16:17:04.714015 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: [pid: 80154|app: 0|req: 1302/2636] 10.222.0.46 () {60 vars in 1365 bytes} [Thu Jun 23 16:17:04 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 23 16:17:14.710431 dv-cinder-835384-9-iscsi devstack@keystone.service[80154]: DEBUG dbcounter [-] [80154] Writing DB stats keystone:SELECT=40 {{(pid=80154) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}